After VMware, VirtualBox provides a fairly nice means of exposing 3D/OpenGL acceleration to guest virtual machines when installing their own custom-developed driver. The guest driver passes the 3D calls to the host to then be accelerated on the hardware. Right now their VirtualBox guest driver isn't Mesa/Gallium3D-based while VMware has its own Gallium3D driver.
It now seems Oracle is investigating creating its own Mesa driver. Michael Thayer, an Oracle engineer, wrote on the Mesa mailing list about a Mesa driver for VirtualBox.
I am looking at the possibility of writing a driver for VirtualBox (ahem, yes) which could partly live inside of the Mesa tree. My initial idea was to have the driver in two parts, a driver/client part in the Mesa tree and a server part in the VirtualBox tree which would communicate using an agreed-on protocol through a local socket. The reasons for wanting to split the driver this way are on the one hand that we still don't feel quite ready to commit our host-guest 3D interface to eternity, and on the other that the driver part on the Mesa side might well be useful in other contexts - controlled 3D access (insofar as such a thing is possible) out of a sand box comes to mind.Interesting, and hopefully it will bear fruit. Right now VMware is much faster than VirtualBox in our Linux OpenGL tests. Meanwhile, for Linux QEMU/KVM we are still waiting for any form of 3D hardware acceleration support for virtualized guests.
I was considering a textual protocol between client and server which would more or less model the Gallium3D API, with (texture and other) buffers passed using shared memory via file descriptors. Gallium3D would have the advantage that it is a much smaller target surface (if I may use that word) than OpenGL. It has the disadvantage that it is closer to D3D than to OpenGL, which may be better for modelling hardware, but perhaps less so for proxying OpenGL APIs on a guest to OpenGL APIs on a host. In particular the fact that we can't directly access the GLSL shader source which applications on a guest system are presumably trying to send us, and which we need to pass through to the host, disturbs me somewhat. Of course, I realise that on the other hand this potentially gives us access to shaders in different formats, but our main target will still be OpenGL on OpenGL for the immediately forseeable future. I understand that Gallium3D receives shaders in a format close to compiled D3D shaders, which suggests that WineD3D might be of use. An alternative is to go straight for a DRI driver rather than a Gallium3D one. I would be interested in any thoughts and suggestions!