There is more to this than only the window managers. But indeed ( Linux, Renoise And Choppy Sound ) window managers can be quite a PITA when it comes to audio performance.
But also Pulse audio doesn’t seem to be the most perfect audio engine to use for real time audio unfortunately.
I’m not currently eager to drop Ubuntu Studio 7.04 or 7.10 and that is quite some versions back, but these use ALSA and Jack solely without the whole load of crap we don’t need (including Pulse audio).