Seeing NVIDIA so quick to put out a GL4.4 driver is hardly a surprise and it's generally been that way for recent OpenGL releases given their involvement with the Khronos Group. The newly released Linux driver to support OpenGL 4.4 and version 4.40 of the GL Shading Language is the 325.05.03 Beta.
In terms of NVIDIA hardware that's capable of hitting all OpenGL 4.4 functionality, you will need a NVIDIA Fermi (GeForce 400/500) or NVIDIA Kepler (GeForce 600/700) series graphics card or a similar workstation GPU from the Quadro series. However, some of the older NVIDIA GPUs only capable of OpenGL 3.x are exposed to some of the new OpenGL extensions where capable.
GL3-capable NVIDIA hardware with this driver can at least get ARB_enhanced_layouts, ARB_multi_bind, ARB_texture_mirror_clamp_to_edge, ARM_texture_stencil8, and ARB_vertex_type_10f_11f_11f_rev. OpenGL ARB extensions limited to GeForce 600 (Kepler) and newer are meanwhile ARB_bindless_texture and ARB_seamless_cubemap_per_texture. There's also new NVIDIA extensions of NV_blend_equation_advanced, NV_bindless_multi_draw_indirect, and NV_gpu_program5_mem_extended.
OpenGL 4.4 / GLSL 4.40 presents buffer placement control for allowing explicit control over the placement of buffers in graphics/system memory, better performance by offering efficient asynchronous queries, the shader variable layout for full control over the placement of shader interface variables, efficient multiple object binding support, and streamlined Direct3D porting support, among other new and improved functionality for graphics programming.
Interested OpenGL fans can fetch the newest NVIDIA beta Windows / Linux / FreeBSD drivers from this NVIDIA.com page. Meanwhile, AMD should soon have out an OpenGL 4.4 Catalyst Beta while it will likely be at least a year (but likely longer) before the Mesa/Gallium3D drivers will be supporting this version of OpenGL.