Each image layer (like you would have in GIMP or Photoshop) takes a couple of MB to store in Java and given the roughly 16MB Java app memory limit you didn't have much memory left to play with after creating 5 layers. I found that apps that used C and the NDK had no such limit; you could allocate huge amounts of memory compared to Java without any issues.
To take advantage of this, I wrote code that would let me allocate C memory, copy a Java bitmap into it then free the Java memory. To make all the app features work without crashes due to Java hitting the memory limit, I would shuffle bitmap memory between Java and C as needed, while having to implement several algorithms in C that I would rather have done in Java. For example, the undo/redo feature needed to store bitmaps of what changed when edits were made so this was mostly kept in C. One tricky issue was I couldn't find a way to get the Java APIs to render image data that was stored in memory used by C; you had to copy the data back into Java first.
I found the above frustrating in that if you're writing an image manipulation app, it means there is a massive incentive to avoid the native Java UI libraries and write your app entirely in C as you don't have the same memory limitations for some reason.