Is there anyway to prevent malicious virtual machines from crashing host machine?

Solution 1:

Never say never. Not possible for general purpose operating systems + hypervisor + hardware to never affect the host. Like any software, complexity means it is going to have flaws, and rarely those are virtual machine escapes or denial of service.

Guest affecting host is a very bad problem, although fortunately rare. The platforms have a strong incentive to fix it, they sell VMs as a stable and secure way to partition compute. Maintaining your systems with updates is the first defense. Modern software has exploit mitigation features, and have patched known flaws.

Mitigating problems that might happen anyway calls for a business continuity plan, as always. Be able to restart instances on different hosts. Test backup restore, enabling a rebuild if necessary.

As a part of strategic planning, think about your platform choices. Several hypervisors exist (Hyper-V, kvm, VMware, PowerVM, bare metal, experimental micro kernels) some of which can run on various CPU architectures (ARM, POWER, x86). Most exploits are platform specific. Diversity of implementations means you have options should one feel insecure or legacy. Admittedly, switching platforms is expensive.

Solution 2:

Let me formulate it in another way: if a malicious VM crashes the host, the VM infrastructure is garbage and has a security hole. It is not possible, it is a CORE function of a VM infrastructure to handle this.