Comment by Waterluvian | Hacker News Reader

Comment by Waterluvian | Hacker News Reader

Parent Full thread

I studied geography for about eight years in university and something that drove me kinda mad is how cartographic authoring tools were more like CAD software than they were like video games (I once had a short rant over dinner with the President of ESRI Canada about how my GPU is built for rendering geometries and ArcGIS ignores its existence).

On the surface, the CAD pedigree makes sense. But… why aren’t more tools like video games? Make it super easy for anyone to start getting involved.

I remember all the gatekeeping-like discussion in “Public Participation GIS” seminars: you’ve gotta be an expert to author good data because there’s so many ways to do it wrong. Blah. No.

One of my daydreams if I get FU money is to make an incredibly user friendly set of web based analytical GIS tools. Not just the basics like buffering and queries, but the entire spatial toolbox to build data processing pipelines. I want a layperson to be able to ask “where should I buy a house if I care about these four parameters?” (and for it to not be a hidden magic purpose built portal on some realtor website)

The immediate retort from my internal pessimist is “the data isn’t available or isn’t consistent“ but I don’t buy that anymore. I remember having to go ask nicely to obtain shapefiles from my city. Decades later it’s all just online. And the community can help catalog data sources, add metadata about transforming them to standard schemas, and sharing data pipelines with the community in some sort of catalog.

I really think it’s a UX problem above all else.

A root cause of the problem you've identified is that the tools are designed to offer first-class support for cartographic applications that pre-date the use of computers. Almost none of this has any relevance to modern geospatial analytics but it pervades the way the software works. Users that just want to build a modern analytic application are still forced to deal with this legacy baggage even though it provides no value to them and often makes their applications worse. The learning curve is unnecessarily steep.

Like you, I am of the opinion that a new set of GIS tools that focus solely on the needs of modern geospatial analysis users would be amazing.

Another gap is that cartographic tools have a deep assumption that geometry does not move. They don't provide support for correctly reasoning about spatiotemporal relationships, which are pretty important in modern geospatial applications.

Your last paragraph resonates strongly with me. I’ve taken my GIS skills and applied it to mobile robotics and performance often falls off a cliff when the moving elements aren’t treated very carefully.

OpenLayers and the rest do a great job these days, though. But a lot of the APIs are clearly intended for set and forget. “Why would you want to update this at 60hz?” I hear it asking me.

Good post. I haven't used Carto, Every Door, nor StreetComplete, but some of the consistency and non-ESRI-ness that OP is seeking seems to me to be present in Regrid fka Landgrid:

https://regrid.com/why-regrid

Isn’t that the point of Carto?

The StreetComplete app for Android is a bit like a game and helps update "trivial" tagging tasks on OSM. I used it a bit while cycling around my hometown. Simply stop, see what you can tag/update, and continue cycling.

There should be more of these simple apps like that

Every Door is really good too (it's Flutter, so iOS/Android). I polished off a shopping centre near here in record time with it.

Links to mentioned apps:

https://every-door.app/

https://streetcomplete.app/

> where should I buy a house if I care about these four parameters?

A lot of this data exists, and is freely available.

I used to work for a startup where we tried to accurately predict the amount of sun a particular plot of land would get. You can pretty much get a 3d map of the world with 5m accuracy (or even more granular, but the datasets get ridiculously big at that point), then render a 360° view from any location in the world and simulate the sun movement. You can even get 3d models of some cities to take buildings into account (the earth model is ground only). The startup eventually failed for Covid reasons.

Satellite data is available for most of the world for things like wind speed, cloudiness, vegetation, ...

If you live in a decent country you may be able to freely get data for zoning, power lines, internet, building density, public transport...

You can build a really good dataset to evaluate a particular plot of land if you combine all this data.

Anyway, hit me up if you ever get that FU money, because I'd also like to do something like this :)

> You can pretty much get a 3d map of the world with 5m accuracy (

Related, maybe OT, but you seem a person that knows a lot more about this than I would know: more exactly I wanted to ask you if you know of a decent enough method to get the height of a building based on GoogleStreetView.

Meaning, I'm "inside" GoogleStreetView, I'm looking at a "building" from outside on the street, I know the geo coordinates of the camera that took the photo, I know the geo coordinates of the building, so that I know the distance between the photo camera and the building, I can "count" (for lack of a better word) the building's size in screen pixels (or some other unit/reference? which one?), and based on all that I'm curious if one can get the real height in meters of said building.

From what I could found around on some other forums there were people mentioning the aperture of the GoogleStreetView photo camera being part of the "equation" to compute it all, and that would be a problem because we do not know the values for the aperture. I have no idea though if all that discussion was on the subject or not.

The images need to be rectified or else, yes, the lens distortion messes up the measurements.

Really you just need a length and an angle to complete the triangle. The length is easy: measure distance from base of building to any point where you know the angle. But getting the angle from a point to the top of the building is tricky. I don’t know if any clever way to estimate that without stereo imagery.

A lot of this stuff is automatically calculated when a digital surface model (DSM) is generated from stereo pairs of images. So you might be able to just look it up in Google Earth. This is the data (probably blended with many other sources such as LiDAR and ground survey data) Google uses to extrude buildings into 3D models.

P.S. this was an assignment in 1st year for me. Estimate the height of the CN Tower from some aerial photos. I feel like I should remember more.

I took a shot at this. If you assume flat terrain between you and the base of the building, the problem can be simplified to basically the distance to the building from your current position times the tangent of the angle from the horizontal plane to the top of the building, from the definition of the tangent function[0]. To find the distance you can plug in your current coordinates and the buildings coordinates in a great circle distance calculator, like this one: [1], or just use the distance measure feature in Google maps. To find the angle you can zoom in on the top of the building in street view mode, and in the URL you'll have the pitch angle as a number between 1 and 179, ending in t [2] (so for instance 150.1t is a pitch angle of 150.1 degrees). Conveniently you'll also have the coordinates of the street view position in the same URL. To get the angle from the horizontal plane you have to subtract 90, since 0 pitch is straight down.

I tried it with a tall building (Turning Torso in Malmö), and got 199 m on my first attempt and 195 on my second. The correct height is 190.4 m, so pretty close. The first attempt was 100 m away, the second more than 1.2 km.

I then tried it with the Chrysler building from the other side of East river about 1.5 km away, and got 274 m which is close to the actual 282 m.

You can probably use more sophisticated methods that downloads the 360 image as a stereographic projection and extracts the pitch angle from there, but for something that's a close enough one off I think this works ok.

[0] https://en.wikipedia.org/wiki/Trigonometric_functions#Right-...

[1] https://www.gpsvisualizer.com/calculators

[2] https://www.trekview.org/blog/2020/decoding-google-street-vi...

> You can pretty much get a 3d map of the world with 5m accuracy

I'm having trouble locating high res DEM files. I can easily find arc second granular data (~30m accuracy at my latitude), but beyond that, it appears it's locked down for academic use or by request only. Where are you getting these freely available datasets?

I'm not a GIS person, I dabble in map stuff for personal use, so forgive me if I'm missing something obvious somewhere.

The lack of high res DEM files angers me a lot as well. 30m simply isn't enough and then there are projects like TanDEM-X where it isn't clear why they can't share the 12m x 12m data with the public if it's mostly payed through public funding.

If you are in the US, the USGS compiles best available data into 3dep. It's pretty amazing when it's recent Lidar (like you can see hiking trails or where they built temporary logging railroads 100+ years ago).

That’s what I’m talking about- but it’s not available to download at sub arc-second resolutions for regular normies like me.

I guess I don't understand. It's a huge, complicated dataset, but I was able to quickly go from here to 1m datasets in my region:

https://www.usgs.gov/3d-elevation-program/about-3dep-product...

Yes, I agree that data balkanization represents a significant hindrance for further development of openstreetmap. Fifteen or so years ago I was experimenting with importing DEM files from the US into Blender for landscaping, and finding the data was frustrating. Sometimes the data was available at a state website, other times at a federal website, sometimes also available on the website of some mapping program/project. Surveys within a given geopolitical constraint would be found on the website of whichever department provided the funding. I understanding that, even if ignoring legal realities, merging this data into a single "ground" (heh) truth would be a herculean task, scientifically. But there was no common database or delivery method. Some departments would link to FTP sites, other would provide arcgis downloads via an interactive web tool. I'd have to depend on user-maintained blog posts to even find the data. And the best data was always locked away behind secure portals that I couldn't get access to.

Nowadays openstreetmap is running into the same problem at the municipality level. In the US, if I understand correctly, publicly-funded geographic surveys are under copyright to whichever governing entity and not eligible for import into openstreetmap.

Take for example the excellent app Every Door (https://github.com/Zverik/every_door) with a focus on easily adding building information. In the US, openstreetmap has no address information. Either that or very little address information - but none in my area. So it'd be easy for me to add building nodes with house numbers but I won't be doing that. Not only is it a massive commitment, but the data is already available at openaddresses.io. The catch? It's "probably" license-compatible with openstreetmap. Before merging, all that data would have to be reviewed by a team of lawyers. That's not something a software developer can help with.

In existing threads here, a top complaint about OsmAnd is that lack of address information. Try planning a route without addresses, and without constantly switching to Google Maps to convert addresses to coordinates. Not very user friendly! Opensupermaps (https://github.com/pnoll1/osmand_map_creation) is a user-maintained merge of openstreetmaps and openaddresses.io, but it doesn't integrate with OsmAnd's data update system.

And that's how data balkanization is breaking openstreetmap, right now.

Is it possible to calculate what the view looks like from any arbitrary coordinate on the planet? At the very least, if there is a view of water, or of trees, or an unobstructed view of the horizon? And quantify that in some simple way?

It's not possible with 100% accuracy, but you might get close. You can model the land easily, but not trees or buildings.

If you're standing on a barren rock it's easy, since there are no obstructions.

If it's a forested area (you get this info from satellite images), you probably can't see water, even though there's no land between you and it, except if you're standing on a cliff or something, so the trees don't bother you.

If it's an urban area, you probably won't be able to determine whether you can see water, since there aren't really models for buildings available.

All the data is there, the ability to quantity it in some simple way is not.

energy-generating systems will always attract money, since they produce something that sells, by definition. Unfortunately you are therefore out of luck.. there will be an endless line of new entries to do what you already did..

on the other side of the coin, is water, flooding and tides. Oceans are rising, many storms are more intense and flood events are increasing. Take your 3D knowledge and apply it to safety for floods and water events. No one wants that work, it is tedious and data-heavy. Yet, that is what is needed for many people who will not or cannot do it themselves. good luck

In addition to StreetComplete which others have mentioned, one of the lightning talks at FOSDEM this year was about a project called CoffeOSM, which allows you to scan your receipt from a coffee shop and add the shop to OSM.[0] It's still just a proof of concept, but hopefully it develops into something more as it's a very cool idea IMO.

[0] https://gitlab.com/netvandal/coffeosm

My guess beyond pure historical baggage is that it is hard to produce user experiences that are simple, powerful and produce consistent data at the same time. Also, niche software will always be worse than top notch game UX.

Games work because they often gradually ramp the difficulty of the problems and the applied solutions up in a very controlled way using tutorial levels, clever world design etc. This means you learn the thing without even thinking you learn it.

It would definitly be possible to do such a thing in a GIS tool and have the user do some fictional work first until they reached a certain level of proviciency and then throw them at the real data.

But without tutorializing either you trust the user and give them powerful and simple tools and live with the broken data they produce (akin to button mashing in a fighting game) or you restrict the data that can be produced and frustrate the users because they cannot get the things done they want to get done.

In the end the quality of UX is always a money thing. A game with bad UX won't sell. A niche GIS software coded by three racoons in a trenchcoat will sell, even of the UX is from the stone age — as long as all other GIS tools are on a similar level.

This is something that I'm doing with MapComplete - certain features are 'unlocked' when a certain number of changes has been reached (e.g. when the contributor has >25 changesets, the tags of the data are shown)

You might be interested in what they are building at http://felt.com/ they have a lot of great data sets already built in and it can handle imports of all kinds of mapping data. Not for actually updating the core data of maps, but seems like the tool to address the UX of mapping multiple types of data and presenting or exploring it.

Woah, your vision sounds very similar to the goals of an open-source web app I started building a few years ago.

https://github.com/SustainableUrbanDesign/app

The idea was inspired by Cities Skylines but I started with a 2D approach since everything is much simpler than with 3D. The idea was to help urban planners fill sustainability gaps in their cities by showing which areas needed access to amenities like bus stops, daycare, grocery stores, and schools.

The approach, as mentioned is to overlay buffers surrounding existing amenities, dissolve those buffers, and find gaps. There can even be an interactive component to show the impact of placing an amenity in a neighborhood, like those emojis that radiate outward when placing a public service in Cities Skylines. The inverse problem can easily be accommodated with the app, helping people find desirable neighborhoods for their next home.

I'd be glad to jumpstart the open-source project if I can find any contributors to help build the project.

I have thought it would be cool to send in swarms of connected drones to do more detailed maps of national parks, with images... First time I thought about it was in the petrified forest in AZ. That it would be cool to actually (virtually) see more than just the path you weren't supposed to leave. But could apply to many things... lots of cameras, sensors and pictures.

RE: Canadian GIS data. I thought it was really cool that most Canadian cities offer LIDAR scans for download by the public

>>One of my daydreams if I get FU money [...]

Is this a situation where an alternate path could be a non-profit foundation soliciting donations and grants to get started?

Also, aren't there a new crop of GIS startups that were funded in the most recent funding cycle, or are they also experiencing "truncated runway" syndrome?

I'd love to work on something like this, maybe after I finish the text editor game I've been making for a few years.

gamification of things is discussed, but it's still not discussed enough (one could even argue that we could reach back to pedagogics ala piaget).

that said there's one aspect about gamification is that it often stops working when information requires deep thinking

but it might still be fixable

i'd love to make everything game like, society as a whole would benefit from it (it could even save energy)

The reason that I haven't pursued a gamification approach with https://mapcomplete.osm.be is twofold:

- I'm focusing on other usability issues (and features) first - I'm worried that "mapping for points" instead of "mapping to have a good map/help the community/map what the contributor needs" will result in lower data quality. Before implementing such a thing, this should be thoroughly studied

True, and even the idea of "points" might be shallow gamification. Creating fun incentives might (I'm now knowledgeable, I'm just a person and I know the power of fun in learning and working) come in other clothes.

Good luck :)

Seen Felt.com? What do you think?

Reply on news.ycombinator.com