Announcement

Collapse
No announcement yet.

Xen - VGA passthrough is the way to go!!!

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • dannysemi
    replied
    Thanks for the reply powerhouse. I've actually learned a lot since my post and what I mentioned in that post wasn't entirely true. I did succeed in achieving hardware acceleration unfortunately I don't think it works as I intended for it. I don't know if you are familiar with Microsoft Hyper-V at all, but I get a similar graphics acceleration to RemoteFX with XCP. I was wondering what your experience with Xen is like. Does the gpu acceleration have limitations, like does it work in fullscreen? Does it fully support your graphics card as if it were in the machine?

    I looked and XCP 1.6 (the version I'm using) uses Xen hypervisor 4.1.3. I think it behaves very differently than what you are experiencing because you are using Xen from the same machine, where as I have XCP installed on a server and I am accessing it from a remote desktop on the LAN with a Gbit connection. I think the differences in behavior can be attributed to the functionality of XAPI but I'm not sure. As far as partial success, I am able to play Borderlands 2 off of Steam in fullscreen windowed mode at nearly max settings. The network lag does make the controls just slightly too slow to really be functional. Most other games give me errors, I think because the driver support isn't there yet.

    I don't know if there is a better way to set this up, or if I can get Xen to function in the same way over the network. The weak link is probably a combination of XAPI and RDP but I'm not an expert.

    As of right now, I'm rating my experience with XCP right behind Hyper-V because with Hyper-V I was at least able to get more games working. XCP is fantastic for whatever basic computing I might want to do, but the purpose of this experiment was to see if I could play games and I would say that, for the moment, has failed.

    FWIW I also tried my Nvidia GeForce 660ti in the server and got identical results.

    I really hope to see this technology advance in the future and give it another shot.

    Leave a comment:


  • powerhouse
    replied
    Originally posted by dannysemi View Post
    Ok, so I've been giving this a shot from a different angle I think without a whole lot of success.

    I installed XCP 1.6 to my home server:
    xeon 3.3 ghz (with VT-d)
    32gb ram
    NVidia quadro 4000
    2 tb hard disk and some other disks

    I manage the server with XenCenter on a basic desktop. This part works beautifully. I can create vm's for days.

    I've experimented with many combinations of os installs and drivers and have gotten mild gpu passthrough success. I installed Win 7 Pro SP1 to a vm and then installed the xs-tools that come with xcp. This gave me a pretty speedy and stable Win7 vm. I enabled gpu passthrough in xencenter and restart my vm and there it is in my device list. I install drivers and restart and everything seems configured properly, my display adapter says NVidia Quadro 4000 without any warnings. But I can't do anything with it. I don't see any improvements since enabling the gpu and I can't get any 3d applications to launch.

    In case you are wondering: I decided to try this method because I'm pretty new to xen. I've been reading forums and the wiki for the last week or so trying to get a handle on it. I went this way because it seemed pretty simple, but obviously its not working the way I want it to. Any help would be appreciated.

    Note: I tried to run 3dmark11 and this is the error I get -

    Workload Single init returned error message: DXGI call IDXGIOutput::GetDisplayModeList failed:

    The requested functionality is not supported by the device or the driver.

    DXGI_ERROR_NOT_CURRENTLY_AVAILABLE
    I'm not familiar with XCP, except that it's the cloud platform version. According to the Xen comparison matrix, XCP lags behind Xen and doesn't give you the latest features.

    I use Xen 4.1.2 with Linux Mint and it works perfect with a Nvidia Quadro 2000.

    Here is a How-to I wrote: http://forums.linuxmint.com/viewtopic.php?f=42&t=112013

    Hope it helps. If not, I'd be glad to help out if I can.

    Leave a comment:


  • dannysemi
    replied
    Ok, so I've been giving this a shot from a different angle I think without a whole lot of success.

    I installed XCP 1.6 to my home server:
    xeon 3.3 ghz (with VT-d)
    32gb ram
    NVidia quadro 4000
    2 tb hard disk and some other disks

    I manage the server with XenCenter on a basic desktop. This part works beautifully. I can create vm's for days.

    I've experimented with many combinations of os installs and drivers and have gotten mild gpu passthrough success. I installed Win 7 Pro SP1 to a vm and then installed the xs-tools that come with xcp. This gave me a pretty speedy and stable Win7 vm. I enabled gpu passthrough in xencenter and restart my vm and there it is in my device list. I install drivers and restart and everything seems configured properly, my display adapter says NVidia Quadro 4000 without any warnings. But I can't do anything with it. I don't see any improvements since enabling the gpu and I can't get any 3d applications to launch.

    In case you are wondering: I decided to try this method because I'm pretty new to xen. I've been reading forums and the wiki for the last week or so trying to get a handle on it. I went this way because it seemed pretty simple, but obviously its not working the way I want it to. Any help would be appreciated.

    Note: I tried to run 3dmark11 and this is the error I get -

    Workload Single init returned error message: DXGI call IDXGIOutput::GetDisplayModeList failed:

    The requested functionality is not supported by the device or the driver.

    DXGI_ERROR_NOT_CURRENTLY_AVAILABLE
    Last edited by dannysemi; 12-11-2012, 06:49 AM.

    Leave a comment:


  • powerhouse
    replied
    Virtualization is the way to go

    Originally posted by mitcoes View Post
    I think XEN VGA Passthorugh MUST BE PRESINTALLED at some computers.

    If any of you knows anyone with a hardware store, or someone at DELL ACER or any brand tell them it would be at the bloggers news a computer with a good XEN VGA passthorught configuration.

    From a AMD Trinity to an Intel I7 any good "XVP" configuration with its MS WOS benchmarks vs baremetal with antivirus and without antivirus would be an excellent product for marketing, giving the brand notoriety, even that specific model perhaps is not a best seller or perhaps

    It is an excellent solution, in a cheap configuration for security at SOHO - Small Offices and HOmes - , and in a expensive one for hard gamers and programmers.
    Yeah, I totally agree. Would be awesome. I had minor nightmares to get my VGA passthrough working. But now that it does, I can't think about anything that comes close to it. Bye bye dual boot. I switch between Windows and Linux at the push of a button. And my Windows performance under Xen with VGA passthrough is just awesome. I doubt it could be any better on bare metal.

    Leave a comment:


  • mitcoes
    replied
    Preinstalled XEN

    I think XEN VGA Passthorugh MUST BE PRESINTALLED at some computers.

    If any of you knows anyone with a hardware store, or someone at DELL ACER or any brand tell them it would be at the bloggers news a computer with a good XEN VGA passthorught configuration.

    From a AMD Trinity to an Intel I7 any good "XVP" configuration with its MS WOS benchmarks vs baremetal with antivirus and without antivirus would be an excellent product for marketing, giving the brand notoriety, even that specific model perhaps is not a best seller or perhaps

    It is an excellent solution, in a cheap configuration for security at SOHO - Small Offices and HOmes - , and in a expensive one for hard gamers and programmers.

    Leave a comment:


  • powerhouse
    replied
    Some more information and minor issues

    VGA and PCI passthrough works well, better than I ever hoped for.

    In Windows domU, I installed the GPLPV signed drivers that can be found here: http://wiki.univention.de/index.php?...-GPLPV-drivers. The disk driver really does wonders - pushed the Windows Experience Index from 5.8 to 7.8.

    I tested my screen calibration software and calibration device on one of the PCI passed through USB ports. It works flawlessly. The software is also able to load the LUT (color adjustment curves) into the screen using the graphics adapter's DVI port.

    Installed Lightroom 4 and CaptureNX2 (Nikon RAW converter) and both work on steroids. It takes only 2-3 seconds (or even less) to load either of them (used to take 10-20 seconds on my old Core2duo rig). RAW photo conversion with adjustments in CaptureNX2 is almost instant (less than 1 second what used to take 10-15 seconds). CPU core temperatures stay lower than 60C during such conversions, when using 5 of 6 cores at 100%. Everything is real snappy, all the while my Linux dom0 runs beside Windows.

    I gave my dom0 some processor scheduling priority, just to be save. When I shut down the Windows domU I can recapture all cores for the Linux dom0. I still need to try ripping some DVDs under Linux to see if that performs as expected (I got above 400 fps without Xen and I hope it will stay like that).

    Minor issues:

    1. For some reason I can't run some "root" applications when clicking on them, for example gparted. However, they work perfectly fine when called from the terminal command line with gksu gparted for example. Synaptic works, though. Gives me a root login window and runs just fine.

    2. Local network file transfers are real slow: 2.3-2.5 MB/s in both Linux and Windows on a 100 Mbps Fast Ethernet wired link. Only around 5.4 MB/s for Gigabit Ethernet. That really sucks! I have a pretty ordinary /etc/network/interfaces configuration where I set up a bridged network. I removed the network-manager package, though, and use a static /etc/resolve.conf file to specify my DNS servers. At first I had a regular setup using network-manager, but when I changed my router and reconfigured my network interface I just couldn't get it to work anymore, only after removing network-manager and manual adjustment / creation of the files specified.

    3. I'm currently copying 800GB from a 1TB drive I took from my old PC onto a Windows domU NTFS file system, from within Linux dom0 using the kpartx utility to mount the domU "partition". It runs at only 40MB/s, pretty slow. The target "partition" is on a logical volume (LV) using two 2TB drives formatted to LVM using stripe which essentially should work like RAID0. So the target should reach a write speed somewhere around 180 MB/s. I would expect to see at least 60MB/s, but 80-90MB/s read speed should be the doable. Which brings me to the question: Which is the best/fastest disk/file access format for domU if I want to access files from both domU and dom0? Currently I'm using phy:/dev/VG-group/LV-name in my Windows domU config file. Is that the best option, or are there better options? I know that people reported file transfer speeds of 80-90MB/s when using the Windows domU machine, but I was hoping to get that on the Linux dom0.
    I think I will try to transfer files from within the Windows domU, just to see how fast that works.

    My idea is to use the Linux dom0 to create snapshots of LVM formatted drives for backup purposes, either when there is no running Windows domU (such as now when I'm transferring the files) or automatically as a cron job while I could be working in Windows. snapshot is a feature of LVM, so it would be running under the Linux dom0.

    Leave a comment:


  • powerhouse
    replied
    Now I got it working - running Windows 7 as a domU with VGA and PCI passthrough

    I finally made it - this time for real !!!

    I had to replace my PNY Quadro 600 (Nvidia) graphics card for a PNY Quadro 2000 card which supports "multi-OS", as Nvidia terms it. It works!

    I have now 2 graphics adapters - a cheap Sapphire (AMD) 6450 card for Linux / dom0 and the Quadro 2000 for Windows 7.

    Windows performance index is 6.8 (the lowest value = storage) - not bad.

    I can shutdown Windows and restart it when needed, without rebooting the Linux dom0. In fact, I can switch between Windows and Linux by the press of 2 buttons: a USB KVM switch to switch keyboard/mouse between 2 USB host controllers (one is passed through to Windows ), and the "input" button on my screen to switch between the two graphics adapters. I could look for a USB+DVI KVM switch to make it even more convenient.

    Anyway, just wanted to let you know.

    The Quadro 2000 card works perfectly, as far as I can tell. I need to install some more software on Windows to give it full trial, including display calibration with a USB screen calibration device and SpectraviewII (NEC software for their screen calibration). I'm right now installing it.

    Will keep you updated and post or link to a how-to, as this is not the easiest thing to get working.

    Note: I didn't have to patch or compile any kernel or xen. I basically used the default packages that came with Linux Mint 13 Mate 64 bit, with some adjustments in the config files and a pciback script, as well as some missing links (ln -s ...) in some places.

    So, the following hardware can be listed as VGA passthrough capable:

    i7 3930K CPU with C2 stepping
    Asus Sabertooth X79 with 1203 BIOS and VT-d enabled
    Sapphire 6450 graphics card for Linux/dom0 (primary card)
    PNY Quadro 2000 (Nvidia) graphics card for Windows 7/domU (secondary card)
    120 GB SSD drive
    Some 2TB hard drives
    Linux Mint 13 Mate 64bit Linux distribution (my setup should also work with other Linux Mint 13 versions and Ubuntu 12.04)
    Xen hypervisor 4.1.2 (default package in Linux Mint 13)
    Windows 7 Ultimate as guest VM / domU

    Thanks for your help and suggestions!

    Leave a comment:


  • powerhouse
    replied
    Mounting a Windows domU LVM partition on the Linux dom0 host system

    For those interested in mounting a domU Windows partition residing on LVM from the Linux dom0, here is what works for me:

    Code:
    sudo apt-get install kpartx
    sudo fdisk -l
    Here the interesting part of fdisk -l:
    Device Boot Start End Blocks Id System
    /dev/mapper/lm13-win7p1 * 2048 206847 102400 7 HPFS/NTFS/exFAT
    /dev/mapper/lm13-win7p2 206848 125827071 62810112 7 HPFS/NTFS/exFAT


    win7p1 is the boot partition, win7p2 is the actual Windows partition which I'm going to mount. lm13 is the volume group I created when installing my Linux Mint system.

    Use kpartx to map the new LVs:
    Code:
    kpartx -av /dev/mapper/lm13-win7
    Note: I didn't try /lm13-win7p2 which may also work.

    Create a mount point and (re-) scan the physical volumes, volume groups, and logical volumes:
    Code:
    sudo mkdir /mnt/win7 # or wherever you want to mount your Windows partition
    pvscan
    vgscan
    lvscan
    Code:
    ls /dev/mapper
    gives me something like this:
    control lm13-root lm13-win7p1 vol1-lm13_backup vol2-swap
    lm13-home lm13-win7 lm13-win7p2 vol2-data


    lm13-win7p2 is what I want.

    Let's mount the Windows file system as NTFS:
    Code:
    sudo mount -t ntfs /dev/mapper/lm13-win7p2 /mnt/win7
    And check:
    Code:
    ls /mnt/win7
    Documents and Settings PerfLogs Recovery Users
    hiberfil.sys ProgramData $Recycle.Bin Windows
    NVIDIA Program Files System Volume Information
    pagefile.sys Program Files (x86) temp


    If the domU is up and running, it might be a better idea to mount the NTFS partition "read-only".

    So I have now full access to the Windows domU file system from within the Linux dom0.
    Last edited by powerhouse; 09-04-2012, 04:30 PM.

    Leave a comment:


  • powerhouse
    replied
    Originally posted by thorgas View Post
    Think about passing a SATA controller through as well.
    This would provide awesome speed and as many HDDs as you like or the controller can handle.
    Thanks thorgas! I was thinking about it, but it doesn't fit my needs. I use LVM for my Linux root and /home partitions, as well as for the Win7 domU. Most of my data needs to be accessed both by Windows and Linux. I haven't tried yet to mount my win7 LV to Linux. I plan to use some disks formatted to NTFS for both Windows and Linux. Need to check that.

    My Win7 domU isn't terribly fast (though it's on a SSD), but OK.

    But first I need to fix this passthrough issue.

    Leave a comment:


  • powerhouse
    replied
    Celebrating too soon - still issues with VGA passthrough

    I was celebrating too soon. While the second graphics controller gets passed through to the Win7 domU, and I can see it listed in the device manager window, I can't get the Nvidia driver (or any driver) to work with the card. So it's marked with a yellow triangle and deactivated by Windows.

    One thing I found is that my Linux distribution (Linux Mint 13, i.e. Ubuntu 12.04-based) uses vpci instead of pass (passthrough) for assigning the PCI in the domU (Windows). That means my physical 02.00.0 and 02.00.1 PCI IDs get changed into different PCI IDs in Windows. But I don't know if that is critical.

    I then decided to give the newer 4.2 Xen hypervisor a try, together with reading the VGA BIOS of the graphics adapter to patch it into the software, as instructed by David Techer here: http://www.davidgis.fr/blog/index.ph...a-pass-through, using the latest 25240 patch as mentioned here http://www.davidgis.fr/blog/index.ph...hes-for-nvidia.

    I managed to pre-compile the tools but got stuck at reading the BAR.

    Code:
    dmesg | grep BAR
    only gives me some BARs for unrelated devices, but not for the graphics card. So I'm stuck.

    Any suggestions on how to read the BAR memory information for the graphics card are most welcome. I already checked through all the log files I can find, and tried lspci -vv.

    I spent much more time than I can afford on this, and I don't want to give up.

    My next alternative is to replace my Nvidia Quadro 600 card for a Nvidia Quadro 2000 card, which Nvidia officially supports as multi-OS capable and supposedly allows VGA passthrough. I found someone who wants to replace his Quadro 2000 for my my Quadro 600 and some $$$ cash.

    But if there is any chance of getting the Quadro 600 to work, it would be great.

    Leave a comment:

Working...
X