Well, that's about to change as in 2.4.2, Pidgin developers has added a shortcut to close the window by using ESC keys. To do that, open Tools > Preferences > Interface. Look at the bottom of the screen and tick the option "Close conversations with Escape key".
Here's the changelog entry:
Add a preference to set Escape as the keyboard shortcut for closing the conversation window.By using this shortcut, it's getting more usable and it saves time (not much though).