Announcement

**Jabberwocky** · 01 December 2017, 10:48 AM

(Reserved for updates)

Update 9 Feb 2018 - Gaming Phase

Great Success

Let start with the results. I'm going to compare the following 3 "systems". Systems A and B are the same physically, but due to the VM config the guest OS refers to it as "AMD Opteron 63xx class CPU" instead of "AMD Ryzen 7 1800X". System C is my old gaming PC which has now been replaced with System A. The systems are running at stock speed, non have been overclocked.

A. AMD Ryzen 7 1800X (Windows 10 VM optimized, 4.14 mainline kernel. Edited QEMU config)
B. AMD Ryzen 7 1800X (Windows 10 VM, stock Ubuntu 17.10 kernel. Config by virt-manager GUI)
C. Intel Core i7-3770 Processor (Windows 10 Native)

Code:

3D Mark Fire Strike:
Points: 11 446 System A
Points: 10 849 System C
Points:  9 853 System B

3D Mark API Overhead Test:
DirectX 11 Multi-threaded draw calls per second
1 471 454 System A
1 171 727 System C
  201 170 System B

DirectX 11 Single-threaded draw calls per second
1 316 305 System A
1 262 224 System C
  202 168 System B

DirectX 12 draw calls per second
18 853 976 System A
17 008 706 System C
 3 837 357 System B

It's interesting to see that Fire Strike (the actual featured benchmark) does not show a big difference between System A and B (basically NPT on/off respectively), however the API Overhead Test shows a massive difference in both DirectX 11 and 12 tests. I do not have Time Spy test at the moment, perhaps it also tests similar aspects. This is the cause of the stuttering I was trying to describe in my 9 Feb 2018 update below.

References:
Fire Strike: https://www.3dmark.com/compare/fs/14...19/fs/14895427
API Overhead Test: https://www.3dmark.com/compare/aot/2...361/aot/248289

I am happy with my current setup! I still think there is room for improvement. I feel like an idiot for assuming Ryzen NPT fixes would be back-ported/not checking what version the patch was accepted into mainline kernel !! I finally have one desktop that can both game and use for software development and Linux is running at the core, no Windows auto update firmware crap. I also managed to workaround my sound issues, I am now passing one of my USB controllers to the VM instead of passing USB devices individually. I'm guessing there a latency or address issue with the Windows sound driver which is related specifically to the Asus Xonar U7 7.1 Usb Soundcard, however my knowledge of what goes on in the hardware/driver level is nonexistent. I might buy an extra PCI-E USB card just for isolated mapping sake, I'm sure BIOS updates can improve mapping too, but I have no idea if/when that will happen.

From here onward, once Linux 4.16 (or better Fedora 28) is released I would like to try https://github.com/gnif/LookingGlass maybe some stability tests in the meanwhile. Ansible script to setup my Host OS could save me time/frustration in the future. I want to take a break from updating pci bus ids in various files and rebuilding grub/initramfs manually. I have not tried overclocking, I'll probably leave that up to some of my friends that I will be assisting with the same setup as this.

A Sad Story,
I found out that I have the compilation segfault issue with All my Ryzen CPUs (I/my colleague bought: x3 1800X ~end of Nov 2017). I was unable to RMA any of those since AMD does not ship to/from South Africa. I know about forward address like Stackry, but based on passed experiences I can't take chances with customs, the risk ridiculous tax is just too damn high. I'll have to find a workaround to the problem in software/config, CPU pinning perhaps? If all else fails I'll have to disable cores until I save up for a replacement or zen+ upgrade. I'm quite baffled that I received chips that were produced in week 05 when I purchased it at the end of 2017, anyway it is what it is.

Update 9 Feb 2018 - Testing Phase Success/Failure

Fail

I made a few attempts at getting Qubes OS working, but I could not install in UEFI mode. Qubes currently uses Fedora 25 for their install. I downloaded F25 and tried UEFI boot, it worked. It was not a upstream issue so I reported it and left it at that. I spoke to the guys on IRC, I was surprised to hear that they are actually looking into gpu passthrough even as one of thier Google Code of Summer projects albeit targeting intel igpu first.

Success

I had success using vfio/iommu pci-e pass-through using plain old Ubuntu. I preferred using Fedora in the past, but I'm using geforce 1030 on my host, proper nouveau support is only landing in ~4.16 and nomodeset's llvmpipe messes up the refresh rate of my screen, depending on the background color I can actually see the screen flickering which hurts my eyes. Nvidia's proprietary driver did not work on Fedora 27's latest kernel. I used the following documentation https://forum.level1techs.com/t/ubun...-14-rc1/119639 there is a lot more people trying it out today and level1techs / reddit has a lot of information and support. I found it easier to setup pci-e passthough than getting geforce 1030 working on my host OS.

This is the hardware I ended up with:
cpu: AMD Ryzen R7 1800X
m/b: ASRock FATAL1TY X370 Professional Gaming
ram: G.SKill F4-3200C14D-32GTZ TridentZ Series 32GB (2x16GB) 3200MHz DDR4 CL14 1.35V
gfx: 8GB Sapphire Nitro+ RX 480
gfx host: 2GB Zotac GeForce GT 1030 ZT-P10300A-10L
ssd: x1 500GB Samsung 960 EVO NVME M.2
hdd: 8TB WD RED

I configured my VM using virt-manager, had to play around with CPU setting as Ryzen is not supported yet with qemu/kvm. I installed windows 10 just had to mount the virtio drivers iso during install. I'm really glad Redhat signed those, it made the process very easy. I installed steam and played a few games.

Play for a hour or so, input lag was worse than native and stuttering was very bad at times. I can still play competitive games like pubg so I'm happy for now. It's a lot better than anything else (waiting for native pupg support or waiting for wine). There are still optimizations that I need to do. I'm going to try out Windows 7 and although most of my iommu groups are really good on the m/b. I don't have an isolated USB controller and for some reason my Asus Xonar U7 7.1 Usb Soundcard does not work when I pass it to Windows 10 using USB passthrough. I know a lot of people use USB headsets that they can pass though without issues. For now I'm stuck with pulseaudio which I really want to avoid. Probably just going to end up putting in a pci-e usb card and pass that through.

I'm still looking forward to test Virgl support for Windows, just because I'm curious not expecting that much of it. Looking Glass also seems like a project with good potential.

6 Dec 2017 - Research Phase

AMD/Ryzen NPT Fix https://www.phoronix.com/scan.php?pa...U-Pass-Through
Headless PCIe Passthrough (25 Nov 2017): https://www.youtube.com/watch?v=1MI1s4hZ_yE

Other (future) solutions:
PreAlpha Virgl Windows Driver: https://gist.github.com/Keenuts/1991...62cf0011147c0b

**Jabberwocky** · 06 December 2017, 05:40 PM

Thanks for the reply debianxfce .

TL;DR My problem is with dual-booting, I consider wine as a step backwards from dual-booting.

I'm kind of happy with gaming on Windows, sure I would prefer it if all my games worked on Linux just as well as it did on Windows. I actively used/modified wine between 2004 and 2011. It took a lot of time to maintain all the different versions with the patches for each game that I used to play. I actually spent more time on wine research/patching/configuration than the amount of reboots and formatting Windows XP needed. I still considered it worth it because I'm very stubborn and everything worked (sometimes faster than on Windows). These days popular competitive games does not work on wine anymore, so it's not really an option in my books even if I had the time.

Logically it makes sense (to me) to focus on pass-through/para-virtualization rather than translating Windows system calls into POSIX-compliant system calls, given that developer time is a finite resource and Window's system calls are still growing

. I still believe Windows' place is in a virtual machine (which is why I posted under software category). For gaming it's not easy at the moment which is why I am trying to find effective solutions in terms of performance and reliability. I'm not so much phased about hardware cost. I'm hopeful that if vfio/iommu pci-e pass-through gaming becomes more popular there will be optimizations in hardware and software to reduce cost.

PS: My Rx 480's silicon is amazing! I'm actually curious if the Nitro+ OC cards are binned. Anyways in Windows it's kicking the 1060's ass, even passing some 1070s on games like rise of the tomb raider, hitman and battlefield 1. Not going to talk about power draw LOL.

Announcement

Upgrades for vfio gaming PC

Upgrades for vfio gaming PC

Comment

Comment