While not dealing with Xen virtualization and a bit off from the original VGA pass-through discussion, pointed out was SPGT, an open-source Single Player Game Transmitter project. This is actually the open-source component to the StreamMyGame.com service. The open-source SPGT has client and server components for running the game with SPGT on one system and then with the SPGT client for being able to connect to the remote system on Windows and Linux systems.
Those wishing to learn more about the SPGT project can visit its SourceForge page. SPGT takes advantage of SDL for the client application, Xvid for compressed image decoding, and CELT for compressed audio decoding.