I was under the impression that there are basically not enough photons at the sizes that most smart phones sensors are. Once you get to low light levels a tiny sensor just can't be fed enough photons to give a good picture.
That's basically it but the amount of light reaching that sensor is improved by a somewhat larger lens than you usually get on a smartphone.
Sure. The more glass you have the more photons you can gather. I have a f1.2 50mm lense and the amount of photons it can pull in is amazing. There is no way any phone is going to be able to replicate this.