John Carmack is realy smart, yet all his tricks rely on the same principle; the trick is in the eye of the beholder. For example side scrolling wasn't possible on PC's because it didn't have the power to update all the pixels, so with Commander Keen he proved that it was possible to only update the pixels that you need to see. With Wolf3D all the way to the Megatexture. He eliminates calculations rather than speed up algorithms and code. Like with lossy audio codecs; strip away what can't be heared/seen easily, or at all.
With ray tracing his trick is indeed culling. Very smart culling like with sparse voxel octrees; reduce geometry to the amount of voxels that matches the amount of pixels that it takes up on the screen. That way you get perfect geometry everywhere, but with the speed of the Wolfenstein3D engine path tracing, but then in 3D.
This is possible with the megatexture technique. His megatexture uses pixels for textures. Voxels are 3D pixels, so he applies that kind of streaming to get awesome image quality, detail and diversity, while limiting the amount of traceable rays.
As far as I'm aware, the ET:QW megatexture was a vastly simplified version of picking out what's needed on screen - I do something similar for terrain rendering, I believe - but Rage is quite a good deal more advanced (probably uses a texture atlas setup - what fun with texture filtering that is).
Then he creates an information tree structure of all the bounces. Probably while doing that he'll stream the colors of all the pixels from the HDD directly to grahics RAM, but that is not certain (my own speculation). Stage two (not speculation) is having shaders 'blit' the colors according to the tree and blend them. This will not be done by the CPU, therefore the CPU can update the world, calculate physics (Carmack lols at GPU shader physics calculations according interviews) and the process starts all over again.
Carmack his trick is not in speeding up the voxel data search while searching data, but reducing the amount of voxel data by streaming in the first place. Then he can create the tree and find colors at breakneck speeds.
The only way that the HDD speed can keep up (or rather the other way around) is by having multiple compressed files, that stream compressed to the CPU RAM, decompress there and at some point be recompressed (if texture/geometry tiles change) and send back to the HDD to be stored.
Given the time it will take for id Tech 6 to finnisch (judged by the time all his previous work took), hardware will be great enough to leverage good looking Monte Carlo calculations and unlimited detail at 30-60fps.