While this sounds reasonable, PA will get better. It has zero copy API so the latency will not grow significantly on localhost. You should definitely use RT kernel and JACK as a backend, but PA can be used as a frontend with low latency overhead. It's just about passing pointers.That's a derail and I don't need to. It's a well known fact among anyone working with audio on Linux, and you can test this at home, easily. Install any of the Linux audio softwares (Ardour, LMMS, Qtractor, all are open source and GPL-licensed, take your pick) and try to use it through PulseAudio. You don't even need a MIDI keyboard, try loading up any kind of instrument and playing with it a virtual keyboard. With PulseAudio, there are noticeable latencies which make playing in realtime a sheer impossibility.
You can try to tweak the PA settings any which way you want, you can't get the sound working glitchlessly with reasonably low latencies. Maybe you can get it to run "good enough" if you're lucky enough to have hardware that works well with PA, but it's not "good enough" for everyone. Now try the same with ALSA or Jack (if you use ALSA, you have to make sure it's routed to bypass PA, otherwise there will be no difference with PA) and you'll see a world of difference. I can easily get around 6ms latencies with ALSA, and that's without doing any tweaks or using RT kernels - with PA it's not even in the same order of magnitude.
PulseAudio is simply not designed for audio work and does not work well in that area. ALSA is a kernel-level system, so of course talking directly to it gives low latency. Whereas, something like Jack is designed by audio professionals, for the very purpose of facilitating low-latency audio. And it's kind of amazing really, Linux can actually be used as a superior proaudio workstation OS, when you choose and setup your hardware correctly - with a RT kernel + Jack, you can get latencies lower than on any other platform, including OS X. And Jack provides a modular design (real modular, not systemd-modular, hehehehe I'm trolling) that's quite unique in the realm of audio systems - it allows you to route audio between applications with very low latencies, so you can create a virtual sound setup in the same way you'd create one with actual hardware synths. Linux is actually quite capable in the realm of proaudio.
There's a reason why PulseAudio isn't being used by audio applications or audio professionals. And it's not "omg lennart let's boycott this because lennart". It's because PA simply isn't up to the task. Sure, it may be fine for normal desktop use and I never claimed anything different. But that doesn't translate to audio work where the requirements are different.
The best of both worlds approach - for hobbyist, non-proaudio-use - is to use PA with a Jack backend, where PA can be used to support things like desktop, music players etc. and it outputs via Jack, while audio applications can talk to Jack directly. This is something I've been meaning to set up myself (I'm lazy, shut up).