Portable Offline Open Street Map | Hacker News Reader

Portable Offline Open Street Map

news.spatialdev.com

167 pointsjharpster49 comments

Let me explain which is the innovation of it:

There are offline OpenStreetMap clients, but there aren't ways to update said map offilne, or to create "mini-OSM" that later can sync with the main one.

For instance, if you're doing a survey in Amazon with a local community, you would need to make a survey, go back somewhere with internet, sync the data with OSM, download the new file and go back to the local community.

The innovation those guys are making is to create a mini-OSM, so the village could have its own mini-OSM, and later that mini-OSM could be synced to the main one.

They are not the only ones trying to do that, an NGO called Digital Democracy is also trying (https://digital-democracy.org)

Hmm, I wonder what happens when you take a team which is heavily editing in a village, and then multiple team members each have their individual mini-OSM of the same village to sync back? Don't you end up needing to be able to express edits as an OSM equivalent of patches for each edit?

Osm changes are already expressed as atomic changes to each version of an object. Reasonably resolving a conflict where two different edits increment the same version of the same object is a difficult problem (the typical way it is currently done is reject the later edit and force manual review/fixup of the conflict).

If someone adds a road and joins it to an existing road at existing point A, while someone else adds a duplicate of the same physical road and joins it at existing point B, it's going to take a rather aggressive heuristic to automatically discard one of them.

Time of entry is the canonical index.

Your comment reminded me of managing merge conflicts in git (or any distributed source control system)

With conflicts, best way to resolve is a human manually deciding how to merge (which part of commit A & which part(s) of commit B to include) :D

That makes me think that applying a machine learning algorithm to suggest conflict merge fixes would be a fun little project.

I wonder if smartphone capacities grew we would one day hit a point where most meaningful data is already preloaded on every device: A full copy of Wikidata, Wikipedia, OpenStreetMap, dictionaries.

The trend goes the complete opposite direction: the devices get faster but we only use it to draw the data from our servers faster. We push all our data into the cloud, although our devices share a private network most of the time.

Providers of the cell phones want to charge you for your data usage over their network. They aren't going to pre-load the most commonly accessed data on the phones.

I don't get my smartphone from a carrier. I get it from a manufacturer. The carrier is a commodity.

We are the minority. Most people have a cellphone because it's price is heavily subsidized. They also only use it for calls, SMS and facebook...

The concern with not having access to data on a device is that the platform controllers are in full control over the data then. Whether it's a control method or simply a connectivity issue, it will be nice to have access to data. Having device differentiation focused on speed of data transfer vs. stored data seems reasonable.

By the current state of the mobile world, you'd have a special Apple vetted copy of Wikipedia with no mention of their competitors, adult themes or anything that could remotely be percieved as offensive to anyone ;)

Except guns. Guns would be included. Because everyone follows American culture.

Because only guns kill:

http://www.mirror.co.uk/news/world-news/teacher-who-allegedl...

If manufacturers would stop removing the SD card slots we might already be at that point. What do you consider a "full" copy of OSM and how big is that?

~44GB compressed. http://wiki.openstreetmap.org/wiki/Planet.osm

That was in reach of my Nokia N900 in 2009. 32GB internal storage with a 32GB SD card. We've gone backwards*

* Yes, it'd fit on any 128GB device, I guess.

Ah, the 29.3GB PBF might be better, I don't know about the CPU/RAM requirements of it though.

But in a Postgres + PostGis it is more than 300GB, and you can add a tile cache or it will be too slow.

Agree, that's uncompressed and with all of the required scaffolding. I would still argue that in the next few years, you'll be able to carry an entirely functional OSM system around with you, tile cache and all (Samsung is predicting 128TB SSDs by 2018).

The offline maps like Osmand and whatnot elide a bunch of data and include indexes, and last I did the arithmetic, were in the vicinity of 50 GB. Numbers here:

http://download.osmand.net/rawindexes/

(Note that there are duplicated areas and wikipedia extracts in there, so a simple sum will be an overestimate)

I've been thinking about this recently. Our phones have so much connectivity yet that allow them to be smarter than us. But you can be dropped in a new location without signal and suddenly you know more than your phone. It's a strange thing.

Yes I believe our phones should have a base level of 'knowledge' stored about our world which doesn't rely on connectivity to a server. Updates could be made either continuously or with each OS update, but they should happen regardless.

The trend is set by mobile phone companies who subsidize and thus influence the hardware that works on their networks and of course Google, who profit from the large datasets stored in Their cloud.

For these trends to reverse right now would require a nonprofit phone manufacturer and people would need to be willing to pay full price for hardware.

It's rarely in the carrier's best interest to reduce demand for their services, but for those who take the initiative this is not only possible, but practical. Android software in particular offers many offline or intermittent connection options. A 128GB microSD these days is affordable.

Maps:Osmand, MAPS.ME News: RSS reader (take your pick) Wikipedia: Kiwix Podcasts:Antennapod

Up and coming distributed technologies like Rumble put in question the absolute necessity of a centralized network at all.

http://www.disruptedsystems.org/

Wikipedia: Aard2 dictionary (Android) http://aarddict.org

The other night I was thinking about the potential for "mapocalypse" -- navigation in a future where paper maps are rare because of services like Google maps and where the network is unavailable indefinitely. Even dedicated navigation devices are forgoing stored maps for connected services.

Over the long term, widespread access to offline maps feels like a critical plan B. I also suspect that we're just at the beginning of a map industry not in the mature commodity phase.

YMMV.

I suspect they're never going to go completely away. I still like to keep a set in my car in case my phone is dead and can't charge it from the car for whatever reason. I believe the Navy is adding celestial navigation back into their curriculum in case their electronic systems are compromised. There's still a lot of use cases where the low-tech solution is a good backup.

You mean like using a wrist watch stars to manually re-enter a space capsule?

https://en.wikipedia.org/wiki/Gordon_Cooper#.22Spam_in_a_can...

yes. Non-commodity, offline maps: http://www.splash-maps.com and by offline I mean can get it wet, indestructible, draw on it, bleed on it, throw it in the washing machine, etc.

There's the wiki printing project, wonder whether there's an osm printing project.

One? Many: http://wiki.openstreetmap.org/wiki/OSM_on_Paper

I guess ink atlas comes closest to trying to create a canonical atlas in paper of the world.

It sounds interesting. But it will be interesting to see how they approach "syncing" the two 'datasets' (they're own, locally modified data, and the main OSM dataset). The current approach with the main OSM editors is to make the mapper manually figure it out.

one thing which would really be great is an OSM client for Kindle/ebook readers with an offline data store. With an optimised interface and map styling it could come in handy in many cases (hiking, bike tours - anything where weight, power and portability are a concern).

Obviously an editor like spatialdev are developing would need more advanced features thus they are targeting android.

You can make a regular book-of-maps for the kindle easily enough. I did it here http://www.kindle-maps.com/

That's really neat. It had never occurred to me that an e-ink screen would be a good medium for a map - well done!

it would be great if you would share how you did that, because there may be other cities and countries than NYC and the UK... ;)

so far i've only used google maps PDFs but that sucks. i've also had good experiences with offline Garmin maps based on OSM data, but that requires a GPS with much lower battery life than a ebook reader... a bunch of providers here: http://wiki.openstreetmap.org/wiki/OSM_Map_On_Garmin/Downloa...

i used this in the past with good success: http://daveh.dev.openstreetmap.org/garmin/Lambertus/

although this seems like the newest best thing: http://garmin.openstreetmap.nl/

I don't understand what this is and where the "Portable Offline Open Street Map" part is described.

There are already tons of applications for offline OSM usage. There are even (sadly) many different vector formats and files available for download. It would be nice to focus improvement of those instead of developing yet another competing standard. I want to be able to use multiple apps with the same data, not having to provide each app with its own format (looking at you, OSMAnd, Oruxmaps, maps.me...).

I cannot see where the "of the world" bit comes into it, but essentially I think it's offline editing.

I'd say it is to have a server with an offline fork, which a) provides you access to OSM without internet (which is the "boring" part) and b) allows you to make changes when you get new data.

When you return to somewhere with better internet, you then can sync the changes to upstream OSM.

And all that prepackaged so it is easy to prepare and deploy to a laptop, a small portable server, ...

a) is "boring", but last I looked was still far from a simple installation (tar.gz, apt-get, docker or otherwise).

On mobile now, so I can't test but if this is a simple way to start a local read only server, it is already a significant improvement.

There isn't a load of benefit to having a read only OSM API database. If you want to load the data into an OSM editor like JOSM, you can open an OSM XML file. If you want to render maps from OSM data, you have to load it into postgresql with osm2pgsql which works with OSM pbf or xml files. A local API db is rarely useful.

Simpler - https://github.com/AmericanRedCross/posm-build seems to be based on Ubuntu 14.04 with some extra packages. There's also a Kickstart script.

Thanks -- that's from the original article, whose contribution describing an easy way to set up a local OSM server is what I was applauding.

Don't they all work with ESRI Shapefiles?

Can you explain in more detail what you are asking?

In general, the Android/ios apps have their own file formats and then have a PC tool for generating those maps. It's relatively trivial to feed a shapefile into those generators (install GDAL/OGR and convert to a format the tool accepts), but there is not really a standard schema in use in shapefiles so it is also necessary to map the attributes in the shapefile over to something the app will understand (or to provide a custom style to the app that is based on the shapefile schema).

My understanding (and I only have a passing knowledge about GIS data formats) is that the ESRI shapefile is the closest to a vector data format that most/all mapping software supports.

I wasn't aware that shapefiles aren't standardised. From the Wikipedia page (https://en.wikipedia.org/wiki/Shapefile ), it looks like this is the official spec:

http://www.esri.com/library/whitepapers/pdfs/shapefile.pdf

Do you have any examples of implementations that differ from this spec?

What I'm talking about would probably fall under "Organization of the dBASE file". Except it's left to the user.

Is a street name given under "FULLNAME" or is is only provided in the parts "PREFIX" "NAME" "SUFFIX". How are various things categorized. Is there a "FUNCTION" column, or a "USE" column. To be useful, software needs instructions how to interpret that information. That's what I meant by a style.

As an example, it might be necessary to map a value of "A41" given in the "CFCC" column to an osm property like "highway=residential". This is a real example from the US imports of TIGER data. A41 means "Local, neighborhood, and rural road, city street, unseparated.", so that mapping is a little bit unfortunate, as in OSM, highway=residential is unlikely for a rural road.

As far as shapefiles being a vector data format that mapping software supports, the device apps aren't really mapping software in that sense, they are too specialized (I think in large part because users of that software expect premade feature styling).

A shapefile is just like a CSV file, but each row also has a geometry shape. The columns are totally arbitrary and could be anything. That's the "shapefile spec". How are roads stored? What about highways/motorways with 2 carriageways? Is that 2 one way lines, or 1 line through the middle? Is the name of the road given in the shapefile? What column stores it? "NAME"? What about the speed limit? "MAXSPEED"? "SPEEDLIMIT"? What units is that speed limit in? mph? kmph? What about having a textual speed limit definition, then you'd have to parse "30 M.P.H." as a speed limit. etc. etc. etc.

Reply on news.ycombinator.com