I have recently switched to a retina Macbook pro and screen wise, some things are totally different there.
Like this situation where guest window of a virtualbox machine is so small that you cannot use it. So small than it is close to the preview thumbnail from VirtualBox manager.
In order to fix this, you need to go to the VirtualBox Preferences » Display and change the Scale Factor to at least 200%. You must stop and start the virtual machine for the modification to kick in.
You can change this also individually for each VM from it's Settings » Display » Screen.
Done!