The biggest issue you have right now is that it's far too easy to completely replace the original room by deliberately or accidentally floodfilling it out of existence.
I'd recommend several possible methods to fix this:
1. Set a MIN_ORIGINAL_IMAGE as a percentage - though it'll have to be a perceptual check since Nano degrades the original image. When it falls below this value, reset the counter.
2. Setup an intermediary LLM to determine if the user's instructions are "malicious"
3. Add system instructions to Nano Banana that coerce user prompts to only allow them to fill in a "portion" of the original image