With Gallium3D, which soon will enter Mesa's mainline code-base, the OpenCL support is relatively easy to come by through just programming a new state tracker. Zack believes LLVM and Clang are the obvious choice over GCC. An OpenCL state tracker has already been started by Zack, but it's not yet open-source -- this though should change soon. What's more interesting is that Zack hopes to have this new state tracker pass at least one conformance test with a minimum of one GPU driver by this summer.
Zack has shared more about his OpenCL thoughts on his blog.