The crudest form of this would be something like Minecraft, where the world is literally made out of cubes. More advanced work would be something like what Ken Silverman did back during the Build engine days, a modern example of which can be seen here: http://advsys.net/ken/voxlap/pnd3d.htm .
The current state of the art appears to be something like voxel quest ( http://www.voxelquest.com/ ) or Atomontage Engine ( http://www.atomontage.com/ ).
~
If you want to learn a bit about voxel engines, Flipcode had a decent series on landscape engines: http://www.flipcode.com/archives/Realtime_Voxel_Landscape_En... .
Basically, the main problem is raycasting into a scene. To learn more about that, I'd look into octrees as a start ( https://en.wikipedia.org/wiki/Octree ).
That's the basic gist. Everything beyond that is optimization.
From: Technical Memo 6 Alvy Ray Smith, July 17, 1995