Letting an agent press a key on the desk, and why nothing reports back

Slightly off-topic for a tracker forum, but it touches the “everything outside the DAW” part of the workflow, so I’ll put it here.

I run Renoise with a small macro pad next to the keyboard. Transport, a couple of pattern switches, one key that mutes the monitor path. Recently I started wiring an agent into that pad, and the interesting problem turned out to be small and annoying.

The setup is a local MCP service that ships with the pad’s software. The agent asks for a list of configured actions, matches one by title, sends a trigger. Three calls, all on localhost. Simple on paper.

Two things bit me:

The agent has no idea what your hardware is. It sees strings. If a key is titled “Button 3”, there is nothing to match against a request — and if two keys are both titled “Mute”, it picks one, silently. I renamed everything to object-plus-action (“Monitor mute”, “Pattern down”) and the guessing stopped.

Nothing reports execution state back. The service sends a trigger, not a confirmation. It does not know whether the action actually fired. So if the pad is unplugged, or the target app has focus elsewhere, you get a successful-looking call and no result. You verify on the device, every time, until you stop trusting it blindly.

I am not replacing Renoise keybindings with this — the pad still does what it did. The only new thing is that an agent can hit the same key a finger can. Curious if anyone here has gone further with agent-driven transport control, because the “no state back” part is what keeps me from using it for anything recording-related.