I thought you were taking about physx going away. I was responding to that, mostly. How it's implemented on any particular system is dependent on what hardware is available, obviously. So, in the not too distant future I'd expect engines to take incorporate opencl to offload the physics add the system is capable. So, if there's only one GPU and a relatively weak CPU it didn't have much choice. If there are a couple of GPUs then you might end up with something like you have in mind.
The point is that just because physx is going away doesn't mean we still lose the ability to do pretty accurate simulations.