Xen - VGA passthrough is the way to go!!!

[salvo2002]

awesome!!

I am so happy I found this thread! I can't believe it is so hard to find info on this, especially when this is extremely useful, just as Lemrouch stated over 2.5 years ago on the xen-devel mailing list: http://old-list-archives.xenproject..../msg00464.html

What is it good for? Think about offices, schools,.. You can save lot of money when you get three computers for price of two or less.

I'd like to thank everyone on this thread for the detailed information they have posted, especially powerhouse (this is the most detailed thread I have found thus far)
The plan:

• Debian or Ubuntu Dom0
• iStar D-300AS (Dom0 & DomU's) just need to pick the hardware....
• Norco DS-12D SAS Enclosure (DAS)
• All this will be in a rack located in a closet

I'll be running three DomU's with VGA passthrough

• Win7 - office computer ~ 30ft from the rack
• WinXP - garage computer for auto repair software ~ 25ft from the rack
• Ubuntu - htpc with mythtv frontend & backend ~ 15 ft from the rack

At least one PV DomU running OpenIndiana to manage the RAID.

I'll post my status here.

[srosato]

Setup guide for Xen HVM VGA Passthrough (Ubuntu + Windows 7) superb performance

Hey guys, I figured after almost one year and a half, that I would post in my installation guide I did a while back while setuping this virtual machine. The machine is still working as of today! It is so very stable it has been running for 6 months without even the need to reboot. Both Win7 and Linux Mint are installed on the same machine and I switch screens using a KVM that plug cables to the two graphic cards that I have. I also have another headless virtual server running in parallel (ubuntu server) and everything just works like a charm. I made a little video a while back demonstrating the performance of such virtualization (I can run games, play HD videos without a problem) and switch between virtual machines. (At that time I had no KVM so I unplugged cables but now its a matter of a keyboard shortcut!)

The installation guide (you might need to adapt, its 1 year and an half old): https://dl.dropboxusercontent.com/u/...assthrough.pdf
The performance demonstration video: http://www.youtube.com/watch?v=KNDFqIJDB-0

Hi guys, sorry for the late reply, I was in the midst of trying all those experiments. It all worked! Huge success story. I am running Kubuntu 12.04 LTS 64 bits as the dom0 and I have win7 64 bits, another Kubuntu 12.04 desktop 64 bits and a ubuntu 12.04 server 64 bits on top as domUs. Win7 and Kubuntu desktop both have a graphical card assigned to them as PCI Passthrough and it works beautifully. Played Diablo 3 (explains the delay), Crysis2, Starcraft 2, all at high graphics without a single lag/crash/reboot/whatsoever in the Win7 virtual machine. So much that I made this project my end school project at my University, so I will definitely post an how to and videos on how I achieved everything. Its pretty simple in the end. Ubuntu 12.04 LTS or any variant are all now based on kernel 3.2 which has built in support for the Xen hypervisor. The only thing that was needed was to recompile the kernel in order to setup the CONFIG_XEN_PCIDEV_BACKEND built in the kernel and not as a module. I did not have any success of grabbing pci devices when using pciback as a kernel module. When this is done, you just have to fallback the PCI devices (I have given a full host controller for win7 and a usb 3.0 controller for linux). The current limitation is that you cannot dynamically assign a PCI-E (graphic card) to a running DomU, in consequence, you need at least two graphical cards (and a third one if you still want output on the Dom0, which I did) that you assign to each system. Afterwards, you unplug your monitor(s) (I have a three monitor setup and it all works super fine - after some configuration of course) and unplug usb mouse and keyboard to pass them to the other system's assigned graphical and USB ports. My next step it to setup a KVM switch that supports dual DVI-D monitors and usb keyboard and mouse (this one does exactly that) in order to switch between system using the console hotkey (see this manual).

My hardware for doing this was of course a VT-d enabled CPU and motherboard:

Asus P9X79, enable vt-d in the bios
Intel Core i7 3820
ATI Radeon HD 6850 (DomU win7)
ATI Radeon HD 6670 (DomU kubuntu desktop)
ATI Radeon HD 6450 (kunbutu Dom0)
OCZ Vertex3 SSD 120GB
24 GB RAM DDR3 1333Mhz (although this sound excessive, I've built this as a workstation since I have servers that will run headless)

I've made this post fast, as I said, I will create an howto in the near future.

[srosato]

Guide to setup Xen HVM with VGA Passthrough

Well it has been one year and a half that I use that setup and it is still working very well! It has been running stable from the very beggining and I even record a 6 months uptime right now! I run Windows 7 and Linux Mint in parallel without any performance issues. I thought it would be the time for me to share my setup guide I did a while back (requires technical knowledge) in order to setup a Xen on ubuntu and use it as dom-0 to install virtual machines using VGA passthrough. I made a performance video back in the days that display Starcraft2 and Avatar HD running on the Win7 virtual machine while still being able to switch to Linux by switching cable. As of today, I no longer switch cables, I use a KVM and a keyboard shortcut and it works very well.

The guide: https://dl.dropboxusercontent.com/u/...assthrough.pdf
The performance video demo (it also display how slow it is using traditional virtualization with Virtualbox): http://www.youtube.com/watch?v=KNDFqIJDB-0

Hi guys, sorry for the late reply, I was in the midst of trying all those experiments. It all worked! Huge success story. I am running Kubuntu 12.04 LTS 64 bits as the dom0 and I have win7 64 bits, another Kubuntu 12.04 desktop 64 bits and a ubuntu 12.04 server 64 bits on top as domUs. Win7 and Kubuntu desktop both have a graphical card assigned to them as PCI Passthrough and it works beautifully. Played Diablo 3 (explains the delay), Crysis2, Starcraft 2, all at high graphics without a single lag/crash/reboot/whatsoever in the Win7 virtual machine. So much that I made this project my end school project at my University, so I will definitely post an how to and videos on how I achieved everything. Its pretty simple in the end. Ubuntu 12.04 LTS or any variant are all now based on kernel 3.2 which has built in support for the Xen hypervisor. The only thing that was needed was to recompile the kernel in order to setup the CONFIG_XEN_PCIDEV_BACKEND built in the kernel and not as a module. I did not have any success of grabbing pci devices when using pciback as a kernel module. When this is done, you just have to fallback the PCI devices (I have given a full host controller for win7 and a usb 3.0 controller for linux). The current limitation is that you cannot dynamically assign a PCI-E (graphic card) to a running DomU, in consequence, you need at least two graphical cards (and a third one if you still want output on the Dom0, which I did) that you assign to each system. Afterwards, you unplug your monitor(s) (I have a three monitor setup and it all works super fine - after some configuration of course) and unplug usb mouse and keyboard to pass them to the other system's assigned graphical and USB ports. My next step it to setup a KVM switch that supports dual DVI-D monitors and usb keyboard and mouse (this one does exactly that) in order to switch between system using the console hotkey (see this manual).

My hardware for doing this was of course a VT-d enabled CPU and motherboard:

Asus P9X79, enable vt-d in the bios
Intel Core i7 3820
ATI Radeon HD 6850 (DomU win7)
ATI Radeon HD 6670 (DomU kubuntu desktop)
ATI Radeon HD 6450 (kunbutu Dom0)
OCZ Vertex3 SSD 120GB
24 GB RAM DDR3 1333Mhz (although this sound excessive, I've built this as a workstation since I have servers that will run headless)

I've made this post fast, as I said, I will create an howto in the near future.

[powerhouse]

@salvo2002:

It seems that you are after something different from what this thread is about. The main idea behind VGA passthrough is to avoid dual-boot - in other words, if you were normally using Linux and sometimes MS Windows (e.g. for games that don't run under Linux), you could install a second VGA card and run Windows in a VM using this VGA card. Of course, this works only on your local desktop PC.

Connecting remote terminals / screens / keyboard/mouse to a server PC is another story. The maximum permissible cable length for HDMI / DVI / whatever is very short, perhaps a couple of feet / 3-4m. The same goes for USB. So if the two "work stations" you like to create are one next to the other, XEN VGA passthrough is a good option. I doubt you can cover the distances from living room to garage using HDMI and USB cables, aside from the technical challenges of installing the cables.

However, you can use Xen also in the "old-fashioned" way. Create multiple VMs for different purposes and let them run on your server. But you would connect to them via Ethernet cables using some remote thin client or low-end PC. Essentially you would run a remote desktop. Read up on X forwarding (for Linux / X windows) such as ssh -X [email protected] (IP.address.of.server meaning the VM you created under Xen, not the dom0). Also look at SPICE and NX that are more efficient protocols for that.

[powerhouse]

@srosato: I had a short look at your Xen HVM with VGA Passthrough installation guide - thanks for sharing. I haven't looked into the details, but do you describe primary passthrough? If not - that is if you describe secondary passthrough - I was wondering why you need to compile the kernel with pciback? Of course there are advantages to having pciback in the kernel, versus loading it as a module (see my how-to on that here: HOW-TO make dual-boot obsolete using XEN VGA passthrough). Or does it have to do with virt-manager?

Any way, it's an interesting approach and can perhaps solve some common VGA passthrough issues, i.e. video driver blacklisting for one.

Thanks in advance for your reply.

[salvo2002]

exactly..

@powerhouse

That is exactly what I am after. I understand what this thread is about and I intend to use VGA passthrough as you've correctly interpreted. I am not worried about HDMI cable length. Blue Jeans Cable has some good info on cable length here: http://www.bluejeanscable.com/store/...hdmi-cable.htm. For USB, I bought some cat5 usb extenders and they work perfect for extending a keyboard and mouse (logitech wireless unifying) It won't support usb hdds or hubs which I don't intend to use anyways.

Running the cables is easy, I already have a 2 inch PVC pipe running from the rack to the attic that was pre-installed when I built my home. This was my "future proofing". I have 2 x cat6 cables running to each room including the garage. I was planning on using this: http://www.snapav.com/p-1072-b-300-hdmatrix-4x4.aspx which has HDMI over cat5e/6 that supports 1080p60 (36-bit) @ 65 ft, and 1080p60 (24-bit) @ 130ft, so in case I can't run a HDMI cable I'll just use the cat6.

I've used NX before. But for this setup I don't want to "remote in". In theory, it's the same setup as your intended purpose, just with the heads in different locations. This should be doable so to speak...

Thanks again for all the info you've posted on VGA passthrough!

@salvo2002:
It seems that you are after something different from what this thread is about. The main idea behind VGA passthrough is to avoid dual-boot - in other words, if you were normally using Linux and sometimes MS Windows (e.g. for games that don't run under Linux), you could install a second VGA card and run Windows in a VM using this VGA card. Of course, this works only on your local desktop PC.

Connecting remote terminals / screens / keyboard/mouse to a server PC is another story. The maximum permissible cable length for HDMI / DVI / whatever is very short, perhaps a couple of feet / 3-4m. The same goes for USB. So if the two "work stations" you like to create are one next to the other, XEN VGA passthrough is a good option. I doubt you can cover the distances from living room to garage using HDMI and USB cables, aside from the technical challenges of installing the cables.

However, you can use Xen also in the "old-fashioned" way. Create multiple VMs for different purposes and let them run on your server. But you would connect to them via Ethernet cables using some remote thin client or low-end PC. Essentially you would run a remote desktop. Read up on X forwarding (for Linux / X windows) such as ssh -X [email protected] (IP.address.of.server meaning the VM you created under Xen, not the dom0). Also look at SPICE and NX that are more efficient protocols for that.

[sonnet]

Sorry to ask, but couldn't find the answer to my question anywhere else: if I have 2 vga, but only one monitor can I passthrough one of the vga to the guest os?
If the answer is yes, then my other question is: does the vga that gets passed to the guest os needs to be physically attached to the monitor ?

[powerhouse]

Sorry to ask, but couldn't find the answer to my question anywhere else: if I have 2 vga, but only one monitor can I passthrough one of the vga to the guest os?
If the answer is yes, then my other question is: does the vga that gets passed to the guest os needs to be physically attached to the monitor ?

Yes to both answers. However, it's not enough to have 2 VGA - your system (CPU, motherboard, BIOS) must support VT-d (Intel) or AMD-Vi (AMD) and it must be enabled. Also, not every VGA card plays nicely with VGA passthrough. Chances are best if you have an AMD (ATI) VGA card to pass through. for more on that, see here: http://forums.linuxmint.com/viewtopic.php?f=42&t=112013.

[jockinator]

yes sonnet, it will work. in my setup I have 1 vga and 1 dvi plug on my monitor. my intel HD4000 is connected with VGA cable and my radeon with DVI.

you can see a video of my working setup here :

http://kdj0c.wordpress.com/2013/02/12/xen-pci-passthrough/

you will see that you also need a USB switch (less expensive than KVM) to pass keyboard/mouse to VM.

[salvo2002]

?Physicalizing? VMs as Desktops

@powerhouse

That is exactly what I am after. I understand what this thread is about and I intend to use VGA passthrough as you've correctly interpreted. I am not worried about HDMI cable length. Blue Jeans Cable has some good info on cable length here: http://www.bluejeanscable.com/store/...hdmi-cable.htm. For USB, I bought some cat5 usb extenders and they work perfect for extending a keyboard and mouse (logitech wireless unifying) It won't support usb hdds or hubs which I don't intend to use anyways.

Running the cables is easy, I already have a 2 inch PVC pipe running from the rack to the attic that was pre-installed when I built my home. This was my "future proofing". I have 2 x cat6 cables running to each room including the garage. I was planning on using this: http://www.snapav.com/p-1072-b-300-hdmatrix-4x4.aspx which has HDMI over cat5e/6 that supports 1080p60 (36-bit) @ 65 ft, and 1080p60 (24-bit) @ 130ft, so in case I can't run a HDMI cable I'll just use the cat6.

I've used NX before. But for this setup I don't want to "remote in". In theory, it's the same setup as your intended purpose, just with the heads in different locations. This should be doable so to speak...

Thanks again for all the info you've posted on VGA passthrough!

Here is a link that explains ?Physicalizing? VMs as Desktops. It's what I am setting up. His setup uses ESXi, I'll be using Xen.

http://thehomeserverblog.com/esxi-lab-specs/