Possible for Midi keyboard to only control instrument when it is selected?

I’m trying to use my midi keyboard to only send midi signal to an instrument when it is selected (highlighted) in the instrument pane. The issue I am experiencing is that I have a midi instrument set up to control my hardware sampler, but it controls it at all times regardless of what instrument I have selected. I can’t control another instrument without activating the sampler too.