Announcement

Collapse
No announcement yet.

Xen - VGA passthrough is the way to go!!!

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #81
    Got it working !!!

    I finally got it working! Thanks to this thread: http://gro.solexiv.de/2012/08/pci-passthrough-howto posted by another member here.

    Some remarks thou:

    1. I used PCI passthrough, not VGA passthrough. The latter gave me only troubles. I couldn't load the xen-pciback module before the nouveaux driver took control of the graphics card. Short of compiling the xen-pciback statically into the kernel, I chose to buy a second graphics card. So now I have one graphics card dedicated to dom0, and another to domU.

    2. I also got myself a KVM switch to connect keyboard and mouse to two different USB2 ports on two different host controllers. My screen has two DVI ports and I can select between the two via button.

    I'm still missing a vital component for my setup - a second DVI-DVI cable which I'll get after the weekend.

    So far I managed to install Win7 Ultimate 64 bit into the domU and it even recognized my 2nd video card. I installed the latest driver for it (via the virt-manager vncviewer under dom0). Windows complains about some problems, but this should hopefully be solved once I'm able to connect my display to the second video card (missing cable, how stupid of me).

    Once everything is fully functional (with the display connected), I report back with more details.

    For those interested, here some system details:

    Asus Sabertooth X79 board
    i7 3930K with C2 stepping, 10 threads for domU, 2 threads for dom0
    AMD 6450 video card for dom0
    Nvidia Quadro 600 card for domU
    32GB memory, 24GB for domU
    120GB Sandisk Extreme SSD
    2x 2TB WD hard drives for data (I need to install 2-3 more drives from my old PC)
    Linux Mint 13 Maya 64 bit Mate edition with Xen hypervisor as dom0
    Win7 Ultimate 64bit as domU

    I use LVM partitions for /root, /home, and win7, as well as for the data drives.

    Now I need to check how I can make hard drives accessible from within the domU. I've got a lot of data to use under Windows. I guess I need to modify my /etc/xen/win7.cfg file.
    Last edited by powerhouse; 31 August 2012, 06:14 PM.

    Comment


    • #82
      Think about passing a SATA controller through as well.
      This would provide awesome speed and as many HDDs as you like or the controller can handle.

      Comment


      • #83
        Celebrating too soon - still issues with VGA passthrough

        I was celebrating too soon. While the second graphics controller gets passed through to the Win7 domU, and I can see it listed in the device manager window, I can't get the Nvidia driver (or any driver) to work with the card. So it's marked with a yellow triangle and deactivated by Windows.

        One thing I found is that my Linux distribution (Linux Mint 13, i.e. Ubuntu 12.04-based) uses vpci instead of pass (passthrough) for assigning the PCI in the domU (Windows). That means my physical 02.00.0 and 02.00.1 PCI IDs get changed into different PCI IDs in Windows. But I don't know if that is critical.

        I then decided to give the newer 4.2 Xen hypervisor a try, together with reading the VGA BIOS of the graphics adapter to patch it into the software, as instructed by David Techer here: http://www.davidgis.fr/blog/index.ph...a-pass-through, using the latest 25240 patch as mentioned here http://www.davidgis.fr/blog/index.ph...hes-for-nvidia.

        I managed to pre-compile the tools but got stuck at reading the BAR.

        Code:
        dmesg | grep BAR
        only gives me some BARs for unrelated devices, but not for the graphics card. So I'm stuck.

        Any suggestions on how to read the BAR memory information for the graphics card are most welcome. I already checked through all the log files I can find, and tried lspci -vv.

        I spent much more time than I can afford on this, and I don't want to give up.

        My next alternative is to replace my Nvidia Quadro 600 card for a Nvidia Quadro 2000 card, which Nvidia officially supports as multi-OS capable and supposedly allows VGA passthrough. I found someone who wants to replace his Quadro 2000 for my my Quadro 600 and some $$$ cash.

        But if there is any chance of getting the Quadro 600 to work, it would be great.

        Comment


        • #84
          Originally posted by thorgas View Post
          Think about passing a SATA controller through as well.
          This would provide awesome speed and as many HDDs as you like or the controller can handle.
          Thanks thorgas! I was thinking about it, but it doesn't fit my needs. I use LVM for my Linux root and /home partitions, as well as for the Win7 domU. Most of my data needs to be accessed both by Windows and Linux. I haven't tried yet to mount my win7 LV to Linux. I plan to use some disks formatted to NTFS for both Windows and Linux. Need to check that.

          My Win7 domU isn't terribly fast (though it's on a SSD), but OK.

          But first I need to fix this passthrough issue.

          Comment


          • #85
            Mounting a Windows domU LVM partition on the Linux dom0 host system

            For those interested in mounting a domU Windows partition residing on LVM from the Linux dom0, here is what works for me:

            Code:
            sudo apt-get install kpartx
            sudo fdisk -l
            Here the interesting part of fdisk -l:
            Device Boot Start End Blocks Id System
            /dev/mapper/lm13-win7p1 * 2048 206847 102400 7 HPFS/NTFS/exFAT
            /dev/mapper/lm13-win7p2 206848 125827071 62810112 7 HPFS/NTFS/exFAT


            win7p1 is the boot partition, win7p2 is the actual Windows partition which I'm going to mount. lm13 is the volume group I created when installing my Linux Mint system.

            Use kpartx to map the new LVs:
            Code:
            kpartx -av /dev/mapper/lm13-win7
            Note: I didn't try /lm13-win7p2 which may also work.

            Create a mount point and (re-) scan the physical volumes, volume groups, and logical volumes:
            Code:
            sudo mkdir /mnt/win7 # or wherever you want to mount your Windows partition
            pvscan
            vgscan
            lvscan
            Code:
            ls /dev/mapper
            gives me something like this:
            control lm13-root lm13-win7p1 vol1-lm13_backup vol2-swap
            lm13-home lm13-win7 lm13-win7p2 vol2-data


            lm13-win7p2 is what I want.

            Let's mount the Windows file system as NTFS:
            Code:
            sudo mount -t ntfs /dev/mapper/lm13-win7p2 /mnt/win7
            And check:
            Code:
            ls /mnt/win7
            Documents and Settings PerfLogs Recovery Users
            hiberfil.sys ProgramData $Recycle.Bin Windows
            NVIDIA Program Files System Volume Information
            pagefile.sys Program Files (x86) temp


            If the domU is up and running, it might be a better idea to mount the NTFS partition "read-only".

            So I have now full access to the Windows domU file system from within the Linux dom0.
            Last edited by powerhouse; 04 September 2012, 04:30 PM.

            Comment


            • #86
              Now I got it working - running Windows 7 as a domU with VGA and PCI passthrough

              I finally made it - this time for real !!!

              I had to replace my PNY Quadro 600 (Nvidia) graphics card for a PNY Quadro 2000 card which supports "multi-OS", as Nvidia terms it. It works!

              I have now 2 graphics adapters - a cheap Sapphire (AMD) 6450 card for Linux / dom0 and the Quadro 2000 for Windows 7.

              Windows performance index is 6.8 (the lowest value = storage) - not bad.

              I can shutdown Windows and restart it when needed, without rebooting the Linux dom0. In fact, I can switch between Windows and Linux by the press of 2 buttons: a USB KVM switch to switch keyboard/mouse between 2 USB host controllers (one is passed through to Windows ), and the "input" button on my screen to switch between the two graphics adapters. I could look for a USB+DVI KVM switch to make it even more convenient.

              Anyway, just wanted to let you know.

              The Quadro 2000 card works perfectly, as far as I can tell. I need to install some more software on Windows to give it full trial, including display calibration with a USB screen calibration device and SpectraviewII (NEC software for their screen calibration). I'm right now installing it.

              Will keep you updated and post or link to a how-to, as this is not the easiest thing to get working.

              Note: I didn't have to patch or compile any kernel or xen. I basically used the default packages that came with Linux Mint 13 Mate 64 bit, with some adjustments in the config files and a pciback script, as well as some missing links (ln -s ...) in some places.

              So, the following hardware can be listed as VGA passthrough capable:

              i7 3930K CPU with C2 stepping
              Asus Sabertooth X79 with 1203 BIOS and VT-d enabled
              Sapphire 6450 graphics card for Linux/dom0 (primary card)
              PNY Quadro 2000 (Nvidia) graphics card for Windows 7/domU (secondary card)
              120 GB SSD drive
              Some 2TB hard drives
              Linux Mint 13 Mate 64bit Linux distribution (my setup should also work with other Linux Mint 13 versions and Ubuntu 12.04)
              Xen hypervisor 4.1.2 (default package in Linux Mint 13)
              Windows 7 Ultimate as guest VM / domU

              Thanks for your help and suggestions!

              Comment


              • #87
                Some more information and minor issues

                VGA and PCI passthrough works well, better than I ever hoped for.

                In Windows domU, I installed the GPLPV signed drivers that can be found here: http://wiki.univention.de/index.php?...-GPLPV-drivers. The disk driver really does wonders - pushed the Windows Experience Index from 5.8 to 7.8.

                I tested my screen calibration software and calibration device on one of the PCI passed through USB ports. It works flawlessly. The software is also able to load the LUT (color adjustment curves) into the screen using the graphics adapter's DVI port.

                Installed Lightroom 4 and CaptureNX2 (Nikon RAW converter) and both work on steroids. It takes only 2-3 seconds (or even less) to load either of them (used to take 10-20 seconds on my old Core2duo rig). RAW photo conversion with adjustments in CaptureNX2 is almost instant (less than 1 second what used to take 10-15 seconds). CPU core temperatures stay lower than 60C during such conversions, when using 5 of 6 cores at 100%. Everything is real snappy, all the while my Linux dom0 runs beside Windows.

                I gave my dom0 some processor scheduling priority, just to be save. When I shut down the Windows domU I can recapture all cores for the Linux dom0. I still need to try ripping some DVDs under Linux to see if that performs as expected (I got above 400 fps without Xen and I hope it will stay like that).

                Minor issues:

                1. For some reason I can't run some "root" applications when clicking on them, for example gparted. However, they work perfectly fine when called from the terminal command line with gksu gparted for example. Synaptic works, though. Gives me a root login window and runs just fine.

                2. Local network file transfers are real slow: 2.3-2.5 MB/s in both Linux and Windows on a 100 Mbps Fast Ethernet wired link. Only around 5.4 MB/s for Gigabit Ethernet. That really sucks! I have a pretty ordinary /etc/network/interfaces configuration where I set up a bridged network. I removed the network-manager package, though, and use a static /etc/resolve.conf file to specify my DNS servers. At first I had a regular setup using network-manager, but when I changed my router and reconfigured my network interface I just couldn't get it to work anymore, only after removing network-manager and manual adjustment / creation of the files specified.

                3. I'm currently copying 800GB from a 1TB drive I took from my old PC onto a Windows domU NTFS file system, from within Linux dom0 using the kpartx utility to mount the domU "partition". It runs at only 40MB/s, pretty slow. The target "partition" is on a logical volume (LV) using two 2TB drives formatted to LVM using stripe which essentially should work like RAID0. So the target should reach a write speed somewhere around 180 MB/s. I would expect to see at least 60MB/s, but 80-90MB/s read speed should be the doable. Which brings me to the question: Which is the best/fastest disk/file access format for domU if I want to access files from both domU and dom0? Currently I'm using phy:/dev/VG-group/LV-name in my Windows domU config file. Is that the best option, or are there better options? I know that people reported file transfer speeds of 80-90MB/s when using the Windows domU machine, but I was hoping to get that on the Linux dom0.
                I think I will try to transfer files from within the Windows domU, just to see how fast that works.

                My idea is to use the Linux dom0 to create snapshots of LVM formatted drives for backup purposes, either when there is no running Windows domU (such as now when I'm transferring the files) or automatically as a cron job while I could be working in Windows. snapshot is a feature of LVM, so it would be running under the Linux dom0.

                Comment


                • #88
                  Preinstalled XEN

                  I think XEN VGA Passthorugh MUST BE PRESINTALLED at some computers.

                  If any of you knows anyone with a hardware store, or someone at DELL ACER or any brand tell them it would be at the bloggers news a computer with a good XEN VGA passthorught configuration.

                  From a AMD Trinity to an Intel I7 any good "XVP" configuration with its MS WOS benchmarks vs baremetal with antivirus and without antivirus would be an excellent product for marketing, giving the brand notoriety, even that specific model perhaps is not a best seller or perhaps

                  It is an excellent solution, in a cheap configuration for security at SOHO - Small Offices and HOmes - , and in a expensive one for hard gamers and programmers.

                  Comment


                  • #89
                    Virtualization is the way to go

                    Originally posted by mitcoes View Post
                    I think XEN VGA Passthorugh MUST BE PRESINTALLED at some computers.

                    If any of you knows anyone with a hardware store, or someone at DELL ACER or any brand tell them it would be at the bloggers news a computer with a good XEN VGA passthorught configuration.

                    From a AMD Trinity to an Intel I7 any good "XVP" configuration with its MS WOS benchmarks vs baremetal with antivirus and without antivirus would be an excellent product for marketing, giving the brand notoriety, even that specific model perhaps is not a best seller or perhaps

                    It is an excellent solution, in a cheap configuration for security at SOHO - Small Offices and HOmes - , and in a expensive one for hard gamers and programmers.
                    Yeah, I totally agree. Would be awesome. I had minor nightmares to get my VGA passthrough working. But now that it does, I can't think about anything that comes close to it. Bye bye dual boot. I switch between Windows and Linux at the push of a button. And my Windows performance under Xen with VGA passthrough is just awesome. I doubt it could be any better on bare metal.

                    Comment


                    • #90
                      Ok, so I've been giving this a shot from a different angle I think without a whole lot of success.

                      I installed XCP 1.6 to my home server:
                      xeon 3.3 ghz (with VT-d)
                      32gb ram
                      NVidia quadro 4000
                      2 tb hard disk and some other disks

                      I manage the server with XenCenter on a basic desktop. This part works beautifully. I can create vm's for days.

                      I've experimented with many combinations of os installs and drivers and have gotten mild gpu passthrough success. I installed Win 7 Pro SP1 to a vm and then installed the xs-tools that come with xcp. This gave me a pretty speedy and stable Win7 vm. I enabled gpu passthrough in xencenter and restart my vm and there it is in my device list. I install drivers and restart and everything seems configured properly, my display adapter says NVidia Quadro 4000 without any warnings. But I can't do anything with it. I don't see any improvements since enabling the gpu and I can't get any 3d applications to launch.

                      In case you are wondering: I decided to try this method because I'm pretty new to xen. I've been reading forums and the wiki for the last week or so trying to get a handle on it. I went this way because it seemed pretty simple, but obviously its not working the way I want it to. Any help would be appreciated.

                      Note: I tried to run 3dmark11 and this is the error I get -

                      Workload Single init returned error message: DXGI call IDXGIOutput::GetDisplayModeList failed:

                      The requested functionality is not supported by the device or the driver.

                      DXGI_ERROR_NOT_CURRENTLY_AVAILABLE
                      Last edited by dannysemi; 11 December 2012, 06:49 AM.

                      Comment

                      Working...
                      X