Raytracing or raymarching. Not sure if there are any games that use it, but raymarching is often used in the demo scene, for example here: http://www.pouet.net/prod.php?which=67433
Only if you use triangles to represent your scene. The (non-realtime) raytracer POV-Ray for example supports several other geometric shapes as an alternative: https://en.wikipedia.org/wiki/POV-Ray#Primitives