You probably need to look into disabling the Android phantom process killer. I had a similar issue and could not get DEs such as XFCE running for a long time and after the fix it was smooth sailing.
There are meanwhile quite a few Google references, the one I used at the time is referenced on this comment in the issuetracker: https://issuetracker.google.com/issues/205156966#comment52