The job of the immediate UI is to just draw the things. Where and how you manage your state is completely up to you.
It seems you assume some sort of OO model.
> When you have a listbox of 10,000 rows and you only draw the visible rows, then the others will lose their state because of this.
Well keep the state then.
Immediate mode really just means you have your data as an array of things or whatever and the UI library creates the draw calls for you. Drawing and data are separate.