Sandy Bridge and VirtualBox

[MoFoQ]

So I wanted to find out if anyone else on phoronix has used virtualbox on a Sandy Bridge based system.

Although, previously, I had it working without incident but lately, any time I enable more than one cpu on a virtual machine (vm), the guest OS will lockup.
I have a "hunch" that it might be related to the power management features of the architecture (hotplug cpu/core) and more specifically, I noticed that it tends to affected more often after the first time the host system has powered down/turned off a cpu (core). A fresh reboot, it's fine until the power management kicks in.
I'm wondering if anyone else as experienced this and if it could be some option in the BIOS that can resolved this.

I've tested it with Virtualbox 4.0.4 OSE and the latest one off of their site.

System specs:
cpu: i7-2600 (not the "K" so I have VT-d)
mem: 16GB
os: ubuntu 11.04 64bit server

Let me know what other info might help and I'll post it

[MoFoQ]

interesting note is that I just noticed that after a while, the kernel takes the other cores offline.

near the beginning of dmesg, I see the Virtualbox kernel module loading and it states it found 8 cores:

[   37.495341] vboxdrv: Found 8 processor cores.
[   37.495822] vboxdrv: fAsync=0 offMin=0x226 offMax=0x20f0f
[   37.495853] vboxdrv: TSC mode is 'synchronous', kernel timer mode is 'normal'.
[   37.495854] vboxdrv: Successfully loaded version 4.0.4_OSE (interface 0x00160000).

then eventually, the other cores stop responding and they get dropped:

[  484.327750] coretemp coretemp.5: TjMax is 99 C.
[  484.540021] CPU 1 is now offline
[  484.540646] CPU 5 MCA banks CMCI:0 CMCI:1 CMCI:3
[  484.541252] coretemp coretemp.6: TjMax is 99 C.
[  484.699387] CPU 2 is now offline
[  484.700668] CPU 6 MCA banks CMCI:0 CMCI:1 CMCI:3
[  484.701334] coretemp coretemp.7: TjMax is 99 C.
[  484.958492] CPU 3 is now offline
[  484.959828] CPU 7 MCA banks CMCI:0 CMCI:1 CMCI:3
[  485.088580] CPU 4 is now offline
[  485.177639] CPU 5 is now offline
[  485.267317] CPU 6 is now offline
[  485.366897] CPU 7 is now offline
[  485.366900] SMP alternatives: switching to UP code
[  485.587697] WARNING! power/level is deprecated; use power/control instead
[ 1310.260218] SMP alternatives: switching to SMP code
[ 1310.267436] Booting Node 0 Processor 1 APIC 0x2
[ 1310.449221] coretemp coretemp.1: TjMax is 99 C.
[ 1310.450157] Booting Node 0 Processor 2 APIC 0x4
[ 1310.456324] Switched to NOHz mode on CPU #1
[ 1310.627090] coretemp coretemp.2: TjMax is 99 C.
[ 1310.627550] Booting Node 0 Processor 3 APIC 0x6
[ 1310.635811] Switched to NOHz mode on CPU #2
[ 1310.806631] coretemp coretemp.3: TjMax is 99 C.
[ 1310.806756] Booting Node 0 Processor 4 APIC 0x1
[ 1310.815302] Switched to NOHz mode on CPU #3
[ 1310.987301] Booting Node 0 Processor 5 APIC 0x3
[ 1310.994788] Switched to NOHz mode on CPU #4
[ 1311.166728] Booting Node 0 Processor 6 APIC 0x5
[ 1311.174275] Switched to NOHz mode on CPU #5
[ 1311.345916] Booting Node 0 Processor 7 APIC 0x7
[ 1311.353758] Switched to NOHz mode on CPU #6
[ 1311.533252] Switched to NOHz mode on CPU #7
[ 2166.098166] coretemp coretemp.5: TjMax is 99 C.
[ 2166.241150] CPU 1 is now offline
[ 2166.242386] CPU 5 MCA banks CMCI:0 CMCI:1 CMCI:3
[ 2166.242946] coretemp coretemp.6: TjMax is 99 C.
[ 2166.390551] CPU 2 is now offline
[ 2166.392364] CPU 6 MCA banks CMCI:0 CMCI:1 CMCI:3
[ 2166.392843] coretemp coretemp.7: TjMax is 99 C.
[ 2166.539962] CPU 3 is now offline
[ 2166.541914] CPU 7 MCA banks CMCI:0 CMCI:1 CMCI:3
[ 2166.659462] CPU 4 is now offline
[ 2166.897341] CPU 5 is now offline
[ 2166.988396] CPU 6 is now offline
[ 2167.077989] CPU 7 is now offline
[ 2167.077993] SMP alternatives: switching to UP code
[ 5467.882727] SMP alternatives: switching to SMP code
[ 5467.889920] Booting Node 0 Processor 1 APIC 0x2
[ 5472.918789] CPU1: Not responding.
[ 5472.918980] Booting Node 0 Processor 2 APIC 0x4
[ 5477.938796] CPU2: Not responding.
[ 5477.938976] Booting Node 0 Processor 3 APIC 0x6
[ 5482.959578] CPU3: Not responding.
[ 5482.959760] Booting Node 0 Processor 4 APIC 0x1
[ 5487.980685] CPU4: Not responding.
[ 5487.980867] Booting Node 0 Processor 5 APIC 0x3
[ 5493.001142] CPU5: Not responding.
[ 5493.001323] Booting Node 0 Processor 6 APIC 0x5
[ 5498.036325] CPU6: Not responding.
[ 5498.036532] Booting Node 0 Processor 7 APIC 0x7
[ 5503.077053] CPU7: Not responding.

a fresh reboot, and it's good....for a short while (/proc/cpuinfo shows all 8 cores initially...but later, only shows one).

It's essentially the "Highlander" effect.....as if the cores have "The Gathering" and disappearing.

[Panix]

You have have VT-d so you should be able to enable IOAPIC. If you have already done so, I don't have another idea yet.

Also, why use VBox OSE? Seems the Sun one is typically used in many cases with less issues but I haven't confirmed yet.

I am merely speculating because I'm interested in using Virtualbox when I next install my next setup. Someone else might have a better idea but hopefully my suggestion helps.

[Panix]

Hey OP, what mobo do you have?

I believe only sandybridge motherboads with the Q67 supports VT-d. So, even though your cpu supports it, you would need a Q67 mobo, FYI.

http://ark.intel.com/compare/52812,52807,52810

http://www.intel.com/support/motherboards/desktop/sb/CS-030922.htm

I just learned this today so if I upgrade to SNB, I'll probably just get a i5-2500k instead of a non-k.

I'm not sure whether Intel is working on revamping the BIOS to allow for the support or whether it's possible. But, you might need to resort to just VT-x for now.

Make sure virtualization is enabled in the mobo BIOS.

[Kano]

Why don't you use vbox 4.1.x?

[MoFoQ]

Motherboard is a H67 based one and it has VT-d as an option in the BIOS (when I was using a 2600K, the VT-d option was greyed out, it was only made available when I got my 2600).
And yes, I do have IOAPIC on.


As for why I fell back to 4.04OSE, 4.12's VBoxHeadless does not have VNC and the RDP option's auth is still borked.

Anyways, after disabling Speedstep in the BIOS, it seems to work fine; very rock-solid.
My guess is that the kernel's support for Speedstep is still not that good.

[Kano]

You did not get that you only need to install the extpack. Install 4.1.x then:

wget -qO- http://kanotix.com/files/install-vbox-extpack.sh|sudo sh

[MoFoQ]

that wasn't the only reason btw; it was to isolate the issue I had.

besides, I prefer to have it updated/managed by apt-get.

[Kano]

The extpack is NOT managed via apt-get for 4.0.x, it just happens that you installed it with 4.0.x and forgot about it. I usally update it on every vbox update, thats why i worte the script. 4.0.x without extpack would lack the same features as you mentioned with 4.1.x. So keep the things correct.

[MoFoQ]

extpack isn't and neither is 4.12 directly from virtualbox's site

anyways, extpack is installed and it's not the cause of the issue and is a tad off topic.