I don’t know of a keyboard-only way, but I see that ⌘W does work after you focus the floating window by clicking on its title bar. Of course, you can also click the tiny red close button to close the window directly.

Hit the same key combo that invoked the character picker (⌘⌃Space in my case).