With the removal of Unity 2D from Ubuntu 12.10 if your hardware/driver doesn't support OpenGL, LLVMpipe will be called in as a replacement. The LLVMpipe driver acts as a software rasterizer that runs off the CPU and can easily drown some modern CPUs with a heavy workload. GNOME developers have also indicated their plans to ultimately use LLVMpipe with the GNOME Shell as their means of fall-back support -- it's already being done this way within Fedora.
With Wayland's Weston, EGL with OpenGL ES is also a hard requirement. The only major desktop not quickly jumping on this bandwagon is KDE with KWin where you will not be forced to use LLVMpipe.
Old Hardware - If you are still running some old hardware that's one decade or more older, it will become increasingly tough to run a modern Linux desktop. If your graphics card or driver doesn't even have OpenGL 2.0 compliance, you will likely be falling back to LLVMpipe. If your GPU is very old, your CPU is also very likely outdated. In that case, the Linux desktop will just be a pain in the ass to run.
In order to get semi-decent results out of LLVMpipe, you need to be running 64-bit Linux, have a multiple processing cores, and ideally be using a CPU that supports modern instruction set extensions like SSE4 and AVX. Any x86 desktop CPU within the past couple of years should be good enough for handling LLVMpipe on a composited desktop if not going overboard on desktop effects, but gaming with LLVMpipe is still basically not possible even with the highest end Intel CPUs.
If you are running old hardware, going forward you will really want to think twice about upgrading your Linux distribution if you do not want to go out and replace your PC.
Virtual Machines - While VMware and VirtualBox guest virtual machines have 3D acceleration support through specially-crafted guest 3D drivers that pass on the calls to the host, this is one of the very sore points right now for the common open-source Linux virtualization stack: QEMU/KVM. There is work towards SPICE with QEMU/KVM to have Gallium3D, nothing has really materialized to date and right now if using QEMU and a Linux guest VM you will be without any form of 3D hardware acceleration.
There is basically no easy workaround to the solution right now of using QEMU/KVM with a Linux desktop that requires LLVMpipe. Either switch Linux desktops or switch to a different Linux virtualization product. If you're running a virtualized Linux desktop day in and day out, I would personally recommend VMware's Linux offerings.
Servers - Most current and past servers have very weak integrated graphics, e.g. the old ATI ES1000 or Matrox or some XGI parts. If you happen to be launching a Linux desktop that then relies upon LLVMpipe, the experience there will be poor too. Fortunately, we are seeing a change in the hardware market that will make things better. Intel CPUs have been getting more powerful integrated graphics, AMD is taking serious their Fusion APU approach with integrated Radeon HD graphics, and GPGPU/OpenCL/CUDA computing is becoming all the more common to the server markets.
With beefier GPUs becoming more common to servers for the general purpose and parallel computing on the graphics processor, worrying about OpenGL requirements on servers will become less of a problem in the future.
ARM - There isn't an open-source Linux graphics driver with OpenGL support for any ARM SoC right now, which means with any Linux distribution you are left with a poor "out of the box" experience. Without the graphics support, LLVMpipe then again pops up, but low-power ARM processors aren't too powerful to begin with.
We are seeing high-performance quad-core ARM hardware becoming more common where it would be easier to deal with LLVMpipe until being able to obtain a binary blob as the ARM graphics driver, but still it's not as fast as x86. The LLVMpipe driver also has yet to see any ARM optimizations, such as being able to better leverage NEON, but I'm hoping Linaro or a similar group will look at optimizing LLVMpipe in the future. Ubuntu 12.10 on the ARM desktop is a wreck due to Unity's new LLVMpipe fallback.
Multi-Seat Computing - Many multi-seat adapters only support 2D graphics, such as the common and well-supported DisplayLink USB adapters for running several keyboard/mouse/display setups from a single PC. Right now these multi-seat systems are falling back to LLVMpipe and it's unlikely we will see any USB-based 3D-capable graphics adapters in the near future.
The hope in this case would be to either be running a desktop that doesn't fallback to LLVMpipe or have a very beefy CPU if powering a number of Linux desktop seats. There is also some hope for the future since the DMA-BUF PRIME GPU offloading work is materializing in mainline. When all the pieces come into place, the Linux graphics stack should be able to pass the rendering of each desktop to a single dedicated GPU on the host and then pass that rendered buffer to the seat's adapter to then be scanned out. Clover state tracker) off the CPU and other interesting concepts.