Why does my guest virtual machine keep freezing?
Check the logs in the Fedora VM to see if anything explains the freeze.
Paging should not be a problem--there's no reason a guest can't page, and paging on the host simply would not freeze the guest.
Please post the version number of VirtualBox you're using. What you are writing about sounds like a VB bug to me rather than any OS problem.