The time it takes between me pressing the shutter button until the camera records the shot as usable data -- this is the important factor. This is what is holding phones back. On phones, I have to wait sometimes up to 0.25-0.5 seconds or even infuriatingly more between the time when I tap the software "capture" reticle and it records the image. This is what is unacceptable in modern phone cameras. By that time the cat has moved, the play is over, my hand has shaken.
On dSLRs this is refreshingly fast. I half-press to focus, complete-press and now I have the image, nearly instantly from a human perception of time, even given the mechanical slowness of the mirror movement. I know the limiting factor is for all practical purposes my own reflexes. For the phone, I know it's crap hardware/software.