Looking for tips: building a workstation around Xen and GPU pass-through
I'm building my new home machine right now and as usual this will be a heavily overloaded workstation that will serve as development and testing machine, file server and game console (plus I'm adding TV cards and multimedia streaming for my home network).
Since these use cases work against each other (file server is unavailable when the machine is used as game console), I'm heavily considering the use of Xen to provide everything at once. Performance shouldn't be an issue in general, but the game console use case definitely requires direct access to a GPU.
Therefore I have several questions:
- is this feasible in general?
- is the GPU passthrough usable enough with non-server GPUs?
- when buying, what technologies should I look for on the motherboard (most likely choosing Gigabyte + Intel CPU again)
- is there some virtualization technology creeping on the horizon that is worth waiting for?
- how good is Xen at dividing resources?
- the file-server and multimedia-streaming portion of the workstation has to work even if the game console part is eating 100% of its resources and vice-versa
- the system will run on top of 8 disks with software Raid6 and LVM,
should I buy a dedicated disk for the game console portion of
the system?
- I have read about some problem with Xen and I/O performance
Any input will be greatly appreciated.
P.S. I won't mind using other technology then Xen, but I'm unaware of any other virtualization platform that supports GPU passthrough.
The information out there about PCI passthrough is terribly outdated and unhelpful.
I have an HD 7850, and a junk PCI vga-only Nvidia card that has to be a decade old. Neither of these devices support Function-Level Reset. They both work flawlessly being passed-through to DomUs; I run my Domain0 headless with no graphics card, both of my cards are sent to pciback at boot up via a kernel argument.
Xen is quite good about assigning resources to a VM and not allowing access to other resources; you can pin your vcpus to pcpus (cores, threads) and assign dedicated memory. Going with a high end Nvidia, you may have some work cut out for you- but it's hard to say since the papers/writeups that say that also say things like Function Level Reset is a necessity: it no longer is.
The critical part in your setup is 3D performance.
I've worked quite a bit with Xen 2-3 years ago - resource allocation worked fine back then and I'm sure that the technology has improved since. Everything but gaming either uses very few resources (file serving, streaming...) or isn't real-time critical (compiling on the dev system, etc).
I havn't done GPU passthrough, but cursory googling doesn't indicate that it works reliably, tweaking is required. Basically, it won't be plug&play - you are pretty much guaranteed that a few/some/many games won't work.
The most reliable option IMHO would be to use Win7 as a host and for games, and run the other services as virtual machines via VirtualPC or VirtualBox. Its not the most elegant solution (everything depends on Windows, every reboot means stopping all the VMs, etc.), but if you want gaming that Just Works, thats the way to go (other than using a second computer).
Is this feasible in general?
Yes. I have rencently performed Xen VGA passthrough and it worked fine. Take a look at the following video: http://www.youtube.com/watch?v=Gtmwnx-k2qg
Is the GPU passthrough usable enough with non-server GPUs?
Yes and no. Officially, only NVDIA Quadro FX 3800, 4800 and 5800 are supported, but I've successfully used a NVIDIA GeForce GTX 460 as in the previous video and Teo En Ming used a GeForce 8400 GS. It really depends on the graphic card.
When buying, what technologies should I look for on the motherboard (most likely choosing Gigabyte + Intel CPU again)
You should make sure the chipset support I/O MMU (AMD-Vi/VT-d), that the CPU support AMD-V/VT-x and that the graphic card support function level reset (FLR).
Note that this technique is not yet mature and it is not guaranteed to work until you try it!