If the goal is to test OS then everything but the OS must be equal.
Running different code from different compiler means we are testing the compiler mostly.
Running different code from different compiler means we are testing the compiler mostly.
Comment