It runs a modified form of 3DS firmware with BSD bits in it. It's not as clear cut as the PS4.
It's looking more and more like it's a modified-but-largely-intact BSD kernel with a 3ds api (not abi?) compatible userspace. Time will probably prove me wrong.