LiDAR scans are finding hidden Roman roads and cutting crime | Hacker News Reader

LiDAR scans are finding hidden Roman roads and cutting crime

wired.co.uk

138 pointsrbanffy77 comments

I really wish this technology could find its way into the hands of Ruth Shady in Peru. She was the discoverer of the Norte Chico civilization, the earliest known civilization in south america, and one of a select few civilizations worldwide that could be called a cradle of civilization (a civilization that emerged without outside influence).

And despite the significance of the discovery, there is almost no funding for continued research, and making basic discoveries of where artifacts might be located is still a major bottleneck.

There have certainly been LIDAR surveys of archaeologically important regions in Peru, but there's a lot more of Peru to scan than the UK.

There is so much land to scan. The vast majority of the Pontic-Caspian steppe for example. There is a lot of history to be found in burial grounds but it's a real lot of sparsely populated land and so noone has the slightest idea where to start.

Oops, Vespasian wasn't Emperor in 43AD, that would have been Claudius. Vespasian was most likely present for the Roman invasion of Britain. He would become Emperor in AD 69.

Cerialis was appointed by Vespasian and fought the Brigantes, so it's more likely the year 43 was wrong and should have been ~71. Vespasian invaded Britannia in 43, probably confused the two.

Has anyone seen any nice examples of work done with the open LiDAR data?

A friend of mine made https://github.com/catenoid/Shadow-Map

Uses UK LIDAR data to show shadow maps. This can be used to show urban areas which are permanently in shadow. Or just as a cool way to map the country.

There's been some work to use it to add metadata to OpenStreetMap:

http://chris-osm.blogspot.com/2015/09/extracting-building-he...

3D printed model of London. http://www.aeracode.org/2016/5/16/hello-london-rising/

That is very cool. I could see lots of people would want one of those on their wall.

Pretty cool. Is there a way a non coder like me could stick in a postcode and get out a 3d file to take for printing?

Finding Digital Surface Models by area (if they exist for the relevant region) on the UK government website isn't too difficult https://environment.data.gov.uk/ds/survey/index.jsp#/survey

The creator uploaded the convert-to-3D program, so once you've found the files it should be possible to turn them into 3D files provide you're comfortable running stuff from the command line and following his advice on parameters. https://github.com/andrewgodwin/lidartile

I haven't tried it yet though - it's on the to do list!

Thanks! Seems very doable...

That's very nice!

I started doing something similar when the Welsh LiDAR data was released ( details in https://theretiredengineer.wordpress.com/category/lidar/ ). I had written some code to manipulate PLY files when I was trying to create 3D models of standing stones using photogrammetry. It was then fairly easy to modify that code to create PLY models using the LiDAR data. However I found two problems for the area that I was interested in ( parts of south west Wales )

  1. The LiDAR data coverage was a bit patchy
  2. There were gaps in the data files which resulted in holes in the model

There was nothing I could do about #1 but it should be fairly easy to fix #2. The other option would be to use a GIS of some sort to process the data.

Anything in particular you want to see?

Wasn't really after anything specific, just some a few interesting things thrown together. Checked back today and there are some good posts above!

Was very helpful in creating a point cloud viewer. Not uk but US open lidar data:

- http://potree.org/potree/examples/ca13.html

- http://potree.org/potree/examples/showcase/lake_tahoe.html

I downloaded all the 1m dsm composite data from the Environment Agency and Natural Resources Wales and used C#, System.Drawing.Graphics and hefty server from AWS EC2 to render 2 million png files, which I then moved onto a Scaleway C1 bare-metal ARM server, to produce this slippy lidar map of England and Wales: https://houseprices.io/lab/lidar/map

This is awesome thanks for sharing

Incredible and you can even see ripples on the sea. I was wondering (as I live by the coast) what the deal was with the sea but I see that because they did multiple passes over the area, the tide is at different heights.

Wow! A heartfelt thank you for this massive effort. Amazing work mate. Cheers!

This is super cool, nice one!

If you're looking for just general examples and inspiration for things that can be done with LIDAR data, the LAStools blog [1] has a lot of em. Their mailing list is interesting as well.

I've been using LIDAR data for doing canopy height modeling, and screening remote areas for possible exceptional/champion trees prior to checking it out in person.

Haven't put anything online yet because I'm very new to GIS and this sort of thing, and am still getting over the horror of what it takes to build a functioning workspace and data pipeline.

("Just take this TIFF and convert it to a CSV" is a phrase I never thought would make sense, but here it does.)

[1] https://rapidlasso.com/blog/

What software are you using? I have found PDAL[0] to be very simple to run with docker, though building from source is a bit consuming, especially as you reach out for more plugins you didn't realize you wanted initially.

PDAL is my current software crush.

[0]http://pdal.io

I was really pleasantly surprised with LAStools. It's mostly closed source, Windows only, and the free version is subtly crippled - but it works really well.

Getting it running under Wine was dead-simple. There's a significant community around it, and the author fights for open formats.

Checking out PDAL is on my todo list for sure.

Mapbox has a tutorial https://blog.mapbox.com/add-lidar-to-mapbox-43f7e3912424 it uses OSS tools and you can host results locally, own server or on Mapbox.

Anyone know where I could license/purchase hi-res LIDAR scans for use in game development?

The national mapping agency of the country you're interested in I would expect.

For the UK, it's the Ordnance Survey (https://www.ordnancesurvey.co.uk)

The UK scans are public domain.

https://environment.data.gov.uk/ds/survey/index.jsp

Just to clarify - they're not Public Domain. They are released under the Open Government Licence - which is compatible with Creative Commons Attribution 4.0 and Open Data Commons Attribution License.

http://www.nationalarchives.gov.uk/doc/open-government-licen...

Depends what you're talking about. Scans of objects? Terrains? Buildings? Different LiDAR data sets for very different tasks. Here is free LiDAR for the state of Oregon: https://gis.dogami.oregon.gov/lidarviewer/

Google Maps is making their data available for gaming now as well[0].

[0] https://developers.google.com/maps/gaming/

While exciting, the barrier seems pretty high to even get access to some of this data. You have to contact their sales department, which probably means it has Enterprise-level pricing.

The data from the Puget Sound LIDAR Consortium [1] is freely available, you just need to ask for an account. (Not 100% sure on licensing terms, though.)

Seattle / King County just finished a rescan in 2016/2017 and the data is stunningly good. [2] Really good consistency and density. The DLM/classification/cleanup seems excellent.

[1] http://pugetsoundlidar.ess.washington.edu/

[2] (private, need an account) http://pugetsoundlidar.ess.washington.edu/lidardata/restrict...

Heh. Good to see people like the products.

Heh, I get the impression you spend a lot of time in Corvallis? ;)

Really though - as a beginner, I started with the 2003 data, and had a really hard time with it. Had to deal with file conversions rerun ground classifications and thin and do all sorts of cleanup to sorta get where I was trying to go.

;-) yep.

opentopography.org has free data throughout the US (point clouds and rasters), but the resolution varies.

I personally [1] have imported it into Unity for a terrain heightmap.

[1] https://github.com/fenwick67/unityterrainfromtiff

Anyone know about LIDAR hardware i can purchase that is great for high resolution at low range. I'm looking for 1-3mm accuracy at a range from 0.1m up to 2m. 2D or 3D. IP67 rated.

LIDARs are awfully expensive for personal use. Look for other 3D scanning methods, and if you still think it's required, rent one.

This isn’t for personal use.

"the power to deploy these technologies at scale will only be deployed to fight serious crime"

[proceeds to use dragnet to crack down on people dumping stuff in the woods]

What's the downside to pervasively prosecuting illegal dumping?

The same as the downsides to using dragnets to prevent any other petty crime.

Littering is a petty crime.

Illegal dumping is scum of the earth stuff.

It opens the doors for going after even smaller crimes. Imagine if all CCTVs in London where online and they used them to purge all the homeless out of the city or jail them. Does that sound like a great idea to you? The law is often flawed and "micromanaging" people can be destructive for society.

So no considering the crime when deciding how to direct resources? Only pursue investigations of some murders?

(murder is the extreme example, but that's the argument you've just made, that devoting resources to prosecuting some particular crime is a slippery slope to abusing the homeless)

Dumping stuff in the woods can be a serious crime. Sometimes you are cycling through an idyllic remote area and you find someone has dumped a lot of asbestos there right there on the trail.

Even if it’s not a seriously dangerous substance, making the landscape ugly like that is still a really shitty thing to do to your fellow citizens. I think it would be great if more dumping violations could be prosecuted.

This tech isn't going to be able to catch people dumping stuff in the woods unless they're doing it on an industrial scale in their own woods to avoid a few million quid in landfill tax.

There is no surveillance of actual people, the government just found a cheap way to find out that their own land had an unexpected change. Finding suspects seems to be old fashioned police work. I find it very hard to find any arguments against this use of the data

Mapping is important application for LiDAR technology. There's going to be a lot of growth in that space https://semiengineering.com/lidar-goes-back-to-the-future/

Is there any repository of lidar data for US? Especially coastal states like Florida, Texas and California?

Yes! Check out the USGS National Map[1].

A lot of the country isn't covered, and what is there is pretty dated, but there's a recent capture of the entire west coast available.

[1] https://viewer.nationalmap.gov/basic

US lidar still remains fragmented across many different portals.

USGS's 3DEEP data for the National Map covers lots of federal and private land. Open Topography holds lots of research data funded by various federal organizations (NSF, NOAA, others) amongst other stuff. State-level Departments of Natural Resources very commonly host survey data.

I have a question about self-driving cars (SDCs).

Some time ago it was mentioned that SDCs can't use deep-learning because it is not sufficiently reliable, and that instead techniques like LiDAR should be used to recognize objects.

However, this made me wonder: how does the car recognize road-signs and traffic lights?

I think you're confusing things. Deep Learning is a method of machine learning. LiDAR is a tool that collects depth data. Using Deep Learning on flat image data to try and detect depth is unreliable, hence SDCs use LiDAR to detect depth. They also use image data from the visible spectrum and beyond to read road signs. etc.

Ok, but what if the deep learning software of the SDC confuses a "no entry" sign with a 100mph sign?

Or is DL classification at 100% accuracy?

I don't think 100mph signs exist, for a start. But the speed limits for sections of road are already known anywhere that a SDC is going to be used. It's an offline database, no sign-reading on the fly required.

The car sends a photo of the sign back to google and a user decodes it in a CAPTCHA.

(I'm mostly joking, but the answer does seem to involve big databases of what the signs and roads are expected to be. I'm not convinced that most of the current "self-driving" solutions can cope with roadworks, traffic cones, diversions etc)

Relevant xkcd: https://xkcd.com/1897/

You can use a combination of sensors. Many use a camera + LiDAR + radar, some use just camera + radar. So... object detection + computer vision. I personally don't think the LiDAR is necessary or feasible.

I believe this talk will specifically and completely answer your questions: https://www.youtube.com/watch?v=LSX3qdy0dFg

Short answer: Self driving vehicles do use deep learning, but they avoid some types not because it is unreliable, but it is processing power intense. Traffic lights, road signs, pedestrians etc.. That's a whole ball of wax to unravel. Check the lecture, its fantastic imo.

Based on this paper, I'd say "poorly".

https://arxiv.org/abs/1707.08945

TLDR: the authors demonstrate that it's possible to apply minor perturbations to signs (for instance, rectangular stickers) that don't significantly change the appearance to humans but cause a classifier to confuse a stop sign with a speed-limit sign.

Self driving cars have lowered some LIDAR costs a factor of 20, with a promise of far more. I wonder if this can translate over into archeology and geology. Imsaw one project at SIGGRAPH using a cheap LIDAR.

If you're surveying on a national scale, then the costs of chartering flights (or driving around with drones) is going to keep costs high in the near future even if the surveying equipment itself plummets in cost

Eventually we'll get drones to do this job. But first LIDAR needs to get small and cheap enough and large drones must be developed to be reliable enough to fly over populated areas.

A lot of the cost of doing LIDAR surveys isn't just the LIDAR sensor itself.

eg, a typical high-quality survey will involve:

- Something like a Leica ALS80 [1]

- A Cessna + good Pilot, the plane needs to be really well instrumented.

- A team of surveyors on the ground to establish ground controls, and occupy points for GPS RTK calculations

- A bunch of post-processing to normalize the data, remove birdies and pits, do ground classifications, etc. (The raw data from the sensor before the processed point cloud is built is really really raw.)

A much cheaper approach which seems to be catching on is UAV photogrammetry. It's processing intensive, but no special sensors are required, and in the end you still get a pointcloud. Sensefly is a company that makes these types of UAVs.

[1] https://leica-geosystems.com/en-us/products/airborne-systems...

For those unaware, all scientific press releases in the UK must express areas in football pitches, and volumes in double-decker buses.

I disagree. IMHO "area the size of Wales" is more commonly used. For everyday magnitudes the nanoWales [1] may be more convenient.

[1] https://www.theregister.co.uk/Design/page/reg-standards-conv...

And this is why people should not take El Reg too seriously.

Or as one incarnation of Joker put it, "why so serious?".

And height in double decker buses as well.

Actually yes, I think I'm wrong: Double Decker Busses is height, and for volume it's Olympic Size Swimming Pools.

Thank God you corrected yourself. Conversion issues like this have derailed spacecraft.

It's funny, but please, let's not turn HN into Reddit ;)

It is unhelpful when it’s 32 million of them, and especially so when the size of a football field is variable.

> the size of a football field is variable

Sure, maybe the local, community field you and me play on, but not the ones where "real" matches are played. They have a standard size.

FIFA for example have the dimensions in their first law: http://www.fifa.com/mm/Document/FootballDevelopment/Refereei...

Probably they mean a standard football field.

It lists a minimum and a maximum on page 34 and it’s a fair range. Am I missing something? If I google the home pitches of a few famous clubs, they all have different sizes too.

Reply on news.ycombinator.com