Is it possible to close the ssh session without closing byobu or the window?
Yes, of course.
You want to "detach" from your session.
You can use Byobu's hotkey, F6, or you can use the tmux escape sequence, Ctrl-a-d.
Full disclosure: I am the author and maintainer of Byobu.