No announcement yet.

Xen - VGA passthrough is the way to go!!!

  • Filter
  • Time
  • Show
Clear All
new posts

  • #51
    GPU Passthrough

    Well I guess according to xensource, hotplugging is not an option for now:

    GPU pass-through API support

    This document contains the software design for GPU pass-through API support. GPU pass-through is already possible at the moment via the VM.other_configci key, which is generally usable for PCI pass-through. This design proposal introduces a new GPU model to the XenAPI through which VMs can be assigned GPUs in a more flexible and convenient way.


    Rather than modelling GPU pass-through from a PCI perspective, and having the user manipulate PCI devices directly, we are taking a higher-level view by introducing a dedicated graphics model. The graphics model is similar to the networking and storage model, in which virtual and physical devices are linked through an intermediate abstraction layer (e.g. the "Network" class in the networking model).

    The basic graphics model is as follows:

    A host owns a number of physical GPU devices (pGPUs), each of which is available for passing through to a VM.
    A VM may have a virtual GPU device (vGPU), which means it expects to have access to a GPU when it is running.
    Identical pGPUs are grouped across a resource pool in GPU groups. GPU groups are automatically created and maintained by XS.
    A GPU group connects vGPUs to pGPUs in the same way as VIFs are connected to PIFs by Network objects: for a VM v having a vGPU on GPU group p to run on host h, host h must have a pGPU in GPU group p and pass it through to VM v.
    VM start and non-live migration rules are analogous to the network API and follow the above rules.
    In case a VM that has a vGPU is started, while no pGPU available, an exception will occur and the VM won't start. As a result, in order to guarantee that a VM always has access to a pGPU, the number of vGPUs should not exceed the number of pGPUs in a GPU group.

    Currently, the following restrictions apply:

    Hotplug is not supported.
    Suspend/resume and checkpointing (memory snapshots) are not supported.
    Live migration (XenMotion) is not supported.
    No more than one GPU per VM will be supported.
    Only Windows guests will be supported.


    • #52
      Originally posted by srosato View Post
      Powerhouse, I have been doing several research lately for the exact same need you have: Linux host with Xen installed and Win7 virtualized with VGA passthrough. I would really much love to play games and use professional photo editing software on a fully virtualized Windows platform on a Linux host with full graphics support. With all the articles I read, it seems to be really feasible with Ubuntu 12.04 LTS based on kernel 3.2 and Win7 as an unmodified guest OS on VT-d supported motherboard and CPU (Intel i5, i7). The only thing that seems impossible and that I'd like to do, is to switch the VGA passthrough onto another graphic card on the press of a keybinding. However, I think the only thing required is running the Linux Dom0 on the internal HD graphic provided by the CPU and have Win7 full control on a more powerful graphic card and unplug the monitors cables (I have three monitors) and plug them to the other graphic adapter (even though a keybinding to switch between the OSes would be better).

      What do you think?

      For the keybinding, I have not experimented with Xen yet, but maybe it is possible to send command through the command line to 'hotswap' the VGA passthrough from one OS to the other, so that the control of the display would switch from one OS to the other? I bet it is not something possible though.
      Sorry for the late reply. I've been trying to install Fedora 16 or 17 for days now but in vain. I found a tutorial for getting a gaming setup on Xen with VGA pass through that strongly suggested Fedora 16. Well, today I managed to finally get a bootable Fedora 17 install but it was totally messed up. My conclusion - never even think about Fedora!!!
      Back to your post. I tried to install LMDE 12 (Linux Mint Debian Edition) on a LVM with Xen and the proprietary Nvidia driver but the Xen kernel didn't give me a GUI. The regular kernel with the Nvidia driver also had some hick ups, though most was OK. All in all Lmde gives a more usable desktop and I will try it again to get a Xen install. Ubuntu might work too, but I feel Lmde is more polished.

      Given the experience now with my new hardware and the ease of using Virtualbox, I will also look into getting VB to run my Win7 and see if I somehow can get it to pass through to the graphics card. I hope I can post some results by the end of the week.
      With regard to Virtualbox I might be able to live with having the color calibration done under Linux and use the Win7 VB install without VGA pass through

      For reference, my hardware is:
      ASUS Sabertooth X79
      i7 3930K
      Sandisk Extreme 120 GB SSD
      WD 2T Green HD and some other HD
      PNY Quadro 600 Nvidia GPU

      I don't have a GPU in my CPU. If I need to I can take a GPU from my old PC.


      • #53
        If yo make it, please make an article

        Thanks for your work.

        If you made it, please post a "how to" article and run some graphic tests at your Nvidia quadro, as 3dmark, Futuremark and phoronix opengl tests between bare metal and VGA passthrought.

        Meanwhile I have read GIMP allows now - 04/05/2012 16 and 32 bits per channel on its development version.


        Today at Libre Graphics Meeting 2012 in Vienna we announced that the development version of GIMP is now capable of processing images in 16bit and 32bit modes, integer or float at your preference.

        Transformation, painting and color adjustment tools will just work in higher bit depth precision modes. More than that, GIMP can load and save 16bit PNG images and save EXR and HDR files now. We also improved support for indexed images, so that you could finally paint over them with the Smudge tool or apply filters.

        There is still a lot of work left to do, and this is a great chance for potential contributors to step up and begin improving the application. Low-hanging fruits include porting of file loaders and savers, filters and other small bits of GIMP that don't require a lot of familiarity with the internal structure. Please contact us in the gimp-developer mailing list.

        Decision on the final feature set in 2.10 is yet to be made, no time-based schedule is available either. However we fully intend to make development cycles much shorter.
        Last edited by mitcoes; 08 May 2012, 08:04 AM. Reason: missspelling


        • #54
          Hello mitcoes: I will be happy to provide an installation howto if and when I manage to get it work. I'm having an important exam these days and had to delay dealing with this. My new PC isn't even connected right now.
          Searching the Internet I found reports that people got VGA passthrough using Xen working under Debian (see, but no detailed description on how to set it up.

          With reference to Gimp, yeah, I heard about 16 or 32 bit support. But in my opinion, this is just the minimum requirement to even consider Gimp as an option. I sure hope that one day Gimp becomes a real alternative to commercial software.

          For me there are other issues that I need to consider. I have tens of thousands of images that I edited using Nikon Capture NX2. Using this software I can undo or redo any editing without loosing anything of the original RAW (NEF) file. Lightroom offers a similar way of editing, but safes the edits in associated Sidekick files. I believe darktable (OSS software) offers similar features. My problem is to get thousands of files into a high quality file format that incorporates the edits and lets me work from there, if needed. Both Lightroom and darktable allow me to open files that were edited with Capture NX2, but of course they don't show any of the edits (just the original picture as captured by the camera, plus their own standard adjustments, if any). This becomes even more complicated when the files also contain Nik filters (an add-on to Capture NX2 and Lightroom/Adobe Photoshop). What makes it even worse is that my Capture NX2 install has troubles batch-processing SOME of the Nik filters, particularly the Pro Contrast filter (it just doesn't show in the processed file).
          In all cases, processing the pictures into an output file would make the edits permanent, without the ability to undo them (using the output file). If I was unhappy with the way I edited a picture, I would have to go back to Capture NX2 to edit the original file and remove part of the edits and redo them (in CNX2 or another software).
          If I were simply to discard all edits and redo them in another software, for example darktable or perhaps Gimp, I would throw away hundreds of hours of editing. Yet, if I continue to use CNX2 I might be walking into a dead end. Nik software / Nikon don't seem to push CNX2, and new releases are rare if any. Adobe and Lightroom have caught up with CNX2 and there are few reasons today to choose CNX2 over the other more popular and vendor-independent options. The only place where CNX2 stands out is the support of camera internal settings (profiles) that are automatically applied when importing/transferring the pictures.

          While I really love to move to Linux based solutions, there are some very good reasons why I need to support legacy Windows stuff, at least until someone comes up with a file converter that allows me to convert or handle CNX2 edited NEF files. If I were a programmer, I would probably already work on such a utility, though it would basically mean to reverse engineer CNX2's way of processing and displaying pictures.

          I will keep you posted about my progress, but I it will take some time.


          • #55
            Originally posted by powerhouse View Post
            Searching the Internet I found reports that people got VGA passthrough using Xen working under Debian (see, but no detailed description on how to set it up.
            I'm one of those people, Djhg2000 in that thread you linked. It's basically the closest I have to a build log, but I can provide you with most of my config files if it would help. I have some studying to do until the end of next week, but I'll reply when I can. Just ask and I'll see what I can do.


            • #56
              Thanks, no need to hurry

              Actually, my computer does not support Xen vga passthrought, not the mainboard, not the processor, and I do not need MS WOS, except for some little things as upgrading my Android tablet the other day . AMD Phenom X3 450 and a Gigabite880GM-UD2H, with excellent price/performance for my normal use, watching videos, surf the web, and some casual gaming.

              it is only i would like to know the how to, because i know it is difficult, and if you do it, It would help a lot the community a blog entry, and i will post it to "elatareao2 and "Usemos Linux", 2 of my favorite Linux blogs in spanish to make an article.

              And in my next computer have a virtual xen dom instead of plugin my old ide HDD when MS WOS is needed.


              • #57
                [QUOTE=powerhouse;259153]mitcoes: Thanks for the reply.
                I did try them. Last time I used Gimp it only supported 8 bits per color channel. /QUOTE]

                Since some major studios using Linux now, i'm sure there is a program for you to do your home-pictures with ;-)




                • #58
                  Success story, VGA passthrough

                  Hi guys, sorry for the late reply, I was in the midst of trying all those experiments. It all worked! Huge success story. I am running Kubuntu 12.04 LTS 64 bits as the dom0 and I have win7 64 bits, another Kubuntu 12.04 desktop 64 bits and a ubuntu 12.04 server 64 bits on top as domUs. Win7 and Kubuntu desktop both have a graphical card assigned to them as PCI Passthrough and it works beautifully. Played Diablo 3 (explains the delay), Crysis2, Starcraft 2, all at high graphics without a single lag/crash/reboot/whatsoever in the Win7 virtual machine. So much that I made this project my end school project at my University, so I will definitely post an how to and videos on how I achieved everything. Its pretty simple in the end. Ubuntu 12.04 LTS or any variant are all now based on kernel 3.2 which has built in support for the Xen hypervisor. The only thing that was needed was to recompile the kernel in order to setup the CONFIG_XEN_PCIDEV_BACKEND built in the kernel and not as a module. I did not have any success of grabbing pci devices when using pciback as a kernel module. When this is done, you just have to fallback the PCI devices (I have given a full host controller for win7 and a usb 3.0 controller for linux). The current limitation is that you cannot dynamically assign a PCI-E (graphic card) to a running DomU, in consequence, you need at least two graphical cards (and a third one if you still want output on the Dom0, which I did) that you assign to each system. Afterwards, you unplug your monitor(s) (I have a three monitor setup and it all works super fine - after some configuration of course) and unplug usb mouse and keyboard to pass them to the other system's assigned graphical and USB ports. My next step it to setup a KVM switch that supports dual DVI-D monitors and usb keyboard and mouse (this one does exactly that) in order to switch between system using the console hotkey (see this manual).

                  My hardware for doing this was of course a VT-d enabled CPU and motherboard:

                  Asus P9X79, enable vt-d in the bios
                  Intel Core i7 3820
                  ATI Radeon HD 6850 (DomU win7)
                  ATI Radeon HD 6670 (DomU kubuntu desktop)
                  ATI Radeon HD 6450 (kunbutu Dom0)
                  OCZ Vertex3 SSD 120GB
                  24 GB RAM DDR3 1333Mhz (although this sound excessive, I've built this as a workstation since I have servers that will run headless)

                  I've made this post fast, as I said, I will create an howto in the near future.


                  • #59
                    Originally posted by Djhg2000 View Post
                    I'm one of those people, Djhg2000 in that thread you linked. It's basically the closest I have to a build log, but I can provide you with most of my config files if it would help. I have some studying to do until the end of next week, but I'll reply when I can. Just ask and I'll see what I can do.
                    Thanks for your offer. Yes I would be happy to get your config files. Just to be sure we are talking about a Debian system?

                    I made some minor progress: LMDE is installed with Xen dom0, virt-manager, and I've created a domU with Fedora 16 just to see that it works. But I believe it's running as a fully virtualized system and not paravirtualized. I'll need to look into that.

                    I'll keep you updated.


                    • #60
                      Hello srosato: congratulations!!!

                      I would really appreciate your help. I've totally messed up my installation and need to start from scratch. Unfortunately it starts already with the OS installation, which doesn't work in UEFI mode, whatever I try. A step by step tutorial would be great, but any guideline as to what the steps are would also be helpful.
                      Hope to hear from you.