Or is the problem different? For example you already have some arbitrary number of windows already displayed at arbitrary locations that you cannot move?
So keep a list of possible candidate positions. Each of these positions represents either an edge of the screen or a position directly above/below/to the left/to the right of an existing window. You will need to calculate different offsets for each of these circumstances to generate candidate positions. You easily eliminate ones that fall outside of the screen.
Each time you attempt to add a window you iterate the list of candidate positions. If a candidate position can fit a window (just use brute force 2D AABB collision detection against all windows, or a quadtree/spatial hash to optimise) then place the window there. If it cannot fit remove the position from the candidate list.
Once you have added the window, calculate all the candidate positions (for each side) and add these to the list of possible positions for later windows.
Well it's late and that's the best I can manage.. I know somebody will point out a flaw here :)
And can you code an AABB tree on a whiteboard? :-)
One could try placing the new window at (100x, 200y) for integer x,y, and check for intersection each time with all the existing windows. 3600 overlap tests in the worst case, but you get huge benefits in practice by hinting the location of the last new window.
Optimize a bit by sorting lists of windows by x and y, so you can binary search for the few candidates to overlap test, and you are down to about 1000 window checks. You miss some cases if windows are very maliciously aligned off grid, but you can mitigate that by using a slightly finer grid.
Or test the 8 neighbors (edges and corners) of each existing window (if any), as it is nearly impossible for available space to be not one of those positions.
(You can compute the next window position after the previous window creation or move, so there is no latency when rendering a new window.)