Hacker News Reader

Have you tried Teardown? Has incredibly good voxel physics. Definitely possible.

Teardown calculates collision meshes and does physics on those. Not on voxels directly, and definitely on in voxel space.