It doesn't have anything to do with saving a few bytes of memory - it's about making the developers more efficient. Letting them add new features quicker, fix bugs more quickly, etc. That's a lot harder to do on a codebase with thousands of entry points, 95% of which are extremely poorly tested and hardly ever used, which has decades of hacks built on top of hacks. That's what the benefit is.
Much of what X does has just been moved into the kernel, libdrm, and mesa anyway, so it's not like anyone should expect wayland to approach X. The whole point of wayland was to get rid of the stuff that nobody used anymore or that had already been moved elsewhere in the graphics stack but was kept around for backwards compatibility requirements.