What is different about some VSTi’s than others that allows me to use heavy DSP’s on one instrument and other or no dsp effects on other aliases, and many VSTi’s cannot and I have to load a whole new instance? Is there a for sure way of getting around the problem outside of loading a whole new instance of the VSTi?
Alias is plugin dependant.
Aliassing usually works with plugin types like sampler vstis.
If a plugin supports channeling, it supports aliassing: that is a mandatory condition for this feature to work and this cannot be circumvented in any way.
Most synth plugins do not support channeling, so usually no alias possibilities for those types.
VST plugins are applications of their own, Renoise can hardly influence their responses if some plugins behave strange.
In cases with heavy DSP plugins that causes severe effects on certain vstis, you might be able to trigger the prebuffering option of those vstis in the Vstcache.xml and see if it improves performance.
Other than that, there is no real way to solve this than informing the plugin developers.
The fact that certain DSP effects do work for various vst instruments says enough about where the root of the problem really lies.