VirtualBox freezes my Host System (Ubuntu 16.04)

When I try to start my guest system, my Xubuntu 16.04 host gets totally freezed and I can't type or move the mouse or do anything. Also the guest system does not start. I recovered an old working snapshot, but that also failed when 97% were loaded. I have to do a hard reset then.

How can I log error messages in that case and how can i figure out why that happens suddenly?

Are you also on the Kernel Version 4.13.0-26-generic ? Apparently there is a problem with that Version and VirtualBox. You could try to install a older Version, but please be aware that it will not contain the most recent Security patches, i.E. for Meltdown.

https://askubuntu.com/a/995308/684252

I was able to replicate this problem. I found the work around here:

https://askubuntu.com/questions/994621/virtualbox-5-0-40-crashes-the-whole-host-since-update-to-kernel-4-13-0-26-generi

I test both solutions discussed, i.e. using an older kernel (4.10.0-28-generic) with VirtualBox 5.0.40 or upgrading to VirtualBox 5.2.6 with the 4.13.0-26-generic kernel. Either solution worked.

I don't have enough reputation to comment, so i have to type here in answers :(

I would start by using Grep to search /var/log/kern.log and /var/log/messages. You can search for the time when the system froze, and see what logs where generated before that. If your certain it was virtual box, you can search for that and see if you can dig any more information on it. There are also log files that are generated within Virtual box that contains information regarding BOTH the host and the vm. That might be a good place to look also if you believe it was the issue.

Check out https://blogs.oracle.com/scoter/virtualbox-log-files-v2 for how to find those logs.

Never-the-less, i would recommend KVM instead of V-Box. Oracle is not updating it as often, and the performance gains of KVM is starting to take a pretty far leap ahead in performance where it no longer makes sense to use V-Box, other than the awesome GUI. Not to mention they currently have some killer security holes known right now, and oracle has yet to update them after some time. Debian even dropped V-box temporarily from its repo because of the lack of responsibility of oracle updating the code.

You can install new version of VirtualBox from its website.
For simplicity, you can follow the accepted answer of this question.
I advise you to install version 5.1 because version 5.2 use Qt 5.6, so it look ugly in Ubuntu 16.04.