Send And Receive Sysex Dumps To/From Outboard Gear

mxb · May 30, 2011, 9:20am

mSepis,

Good luck with the development of this tool. I’m sure you’ll end up learning much more than you think you will at the start!

If I may suggest a basic procedure for recording/saving SysEx:

This could open a GUI dialog that asks the user which MIDI device name / channel to send it on.

This may be an alternative method that would allow a user to have a normal folder structure to organise SysEx files. I’ve done a fair amount of work with file-browser interfacing for the File Formats tool, so I can assist with this integration if you wish. (It’s fairly easy to integrate once the tool functionality is there - I would suggest this is added towards the end).

Good luck!

mxb · June 7, 2011, 8:05pm

mSepsis / anyone else,

Has anyone done any work on this?

I’ve got a quiet weekend coming up and wouldn’t mind spending some time working on this.

I don’t have any external hardware to test with but I can at least make some of the foundations. I’d be aiming to do something like the following:

Enumerating / Binding to midi ports
Send / Receive sysex routines
Saving preferences (last used midi ports etc.)
Loading / Saving .syx format files
Integration into the native file browser

However, I don’t want to tread on anyone elses toes or reduplicate others work. Thus, this post to let everyone else know my current intentions.

esaruoho · June 7, 2011, 8:16pm

I don’t think anyone’s necessarily doing anything.
I did a really minor thing, a currently_selected_instrument’s midi_program change:

function midiprogram(change)  
local midi=renoise.song().selected_instrument.midi_output_properties  
local currentprg=midi.program  
 currentprg = math.max(1, math.min(128, currentprg + change))  
 rprint (currentprg)  
 print ("yo")  
renoise.song().selected_instrument.midi_output_properties.program = currentprg  
renoise.song().transport:panic()  
end  
  
--renoise.song().selected_instrument.midi_output_properties.program =   
-->>> oprint (renoise.song().selected_instrument.midi_output_properties)  
--class: InstrumentMidiOutputProperties  
 --properties:  
 --bank  
 --bank_observable  
 --channel  
 --channel_observable  
 --delay  
 --delay_observable  
 --device_name  
 --device_name_observable  
 --duration  
 --duration_observable  
 --instrument_type  
 --instrument_type_observable  
 --program  
 --program_observable  
 --transpose  
 --transpose_observable  
  
renoise.tool():add_keybinding { name = "Global:Impulse:Next/Prev Midi Program +1",   
invoke = function() midiprogram(1)  
end }  
renoise.tool():add_keybinding { name = "Global:Impulse:Next/Prev Midi Program -1",   
invoke = function() midiprogram(-1)  
end }

That’s all I know about

mSepsis · June 8, 2011, 10:35pm

I have a vbscript that when run in midiox converts the microwave’s sysex dump to controller changes… somewhat related project to this, but no besides buying a few Lua books and reading up and experimenting I haven’t made any real progress to report on building this tool. I’m happy to see there has been some interest and it’s not just me wishing for this in renoise…

here is the vbscript if anyone is interested at all, credit goes to “fozzie” over on the ctrlr forums. I designed, built and coded the ctrlr, he wrote this script.

  
' Create object  
Set mox = WScript.CreateObject("MIDIOX.MOXScript.1", "OnTrigger_")  
  
mox.DivertMidiInput = 0 'no input diversion  
mox.FireMidiInput = 1 ' Start processing MIDI  
MsgBox "Press OK to end MIDI loop" ' Will not return until after [OK]  
mox.FireMidiInput = 0 ' All done processing  
mox.DivertMidiInput = 0  
  
mox.ShutdownAtEnd = False  
  
Set mox = nothing ' Cleanup  
  
  
Sub OnTrigger_SysExInput (strSysex)  
If len(strSysEx) > 100 Then  
sysex = Split(strSysEx)  
  
port = -1  
  
'MsgBox strSysEx  
  
'Conversion of all relevant Sysex parameters to implemented CC messages on channel 1  
strsysexIndexCC = "8 9 10 12 13 14 19 20 21 23 24 25 26 32 33 34 35 36 37 38 43 44 45 46 47 48 49 54 55 56 57 69 70 71 72 73 74 80 81 82 84 86 87 89 91 94 95 96 97 99 100 101 102 103 104 105 106 107 120 121 122 123 124 126 127 128 129 130 156 157 158 159 160 161 162 163 164 166 167 168 169 170 171 173 174 175 176 177 178 179"  
sysexIndexCC = Split (strsysexIndexCC, " ")  
strccIndex = "33 34 35 36 37 13 38 39 40 41 42 43 44 70 71 72 73 74 75 76 77 78 79 80 81 82 83 45 46 47 48 50 56 54 51 52 53 60 61 62 57 58 55 12 10 65 22 23 5 102 105 104 103 107 106 108 109 110 14 15 16 17 29 18 19 20 21 31 85 86 87 88 89 90 91 92 93 24 25 30 112 113 114 26 28 27 115 116 117 118"  
ccIndex = Split (strccIndex, " ")  
For i = 0 to 89  
mox.OutputMidiMsg port, 176, ccIndex(i), cint("&H" & sysex(sysexIndexCC(i)))  
Mox.sleep(10)  
Next  
  
'part two of conversion; the parameters that are not coupled to a CC will be transformed into separate sysex parameter change messages (part 1 for parameters 0-127  
  
strsysexIndexUnmappedChannel1 = "58 60 61 62 64 77 83 88 90 92 93 108 109 110 111 112 115 116 117 119 132 133 134"  
sysexIndexUnmappedChannel1 = Split (strsysexIndexUnmappedChannel1, " ")  
strSysexLoBitIndex = "33 35 36 37 39 46 4C 51 53 55 56 65 66 67 68 69 6C 6D 6E 70 7D 7E 7F"  
SysexLoBitIndex = Split (strSysexLoBitIndex, " ")  
SysexParamChange = Array ("F0","3E","0E","00","20","00","00","00","00","F7")  
  
  
For j = 0 to 22  
SysexParamChange (7)= SysexLoBitIndex(j)  
SysexParamChange (8)= sysex(sysexIndexUnmappedChannel1(j))  
  
SysexParamOutput = Join (SysexParamChange, " ")  
  
mox.SendSysExString SysexParamOutput  
Mox.sleep(10)  
Next  
  
'part three of conversion; the parameters that are not coupled to a CC will be transformed into separate sysex parameter change messages (part 2 for parameters 128-256  
  
  
strsysexIndexUnmappedChannel1_pt2 = "135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246"  
sysexIndexUnmappedChannel1_pt2 = Split (strsysexIndexUnmappedChannel1_pt2, " ")  
strSysexLoBitIndex_pt2 = "00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 2F 30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F 40 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 51 52 53 54 55 56 57 58 59 5A 5B 5C 5D 5E 5F 60 61 62 63 64 65 66 67 68 69 6A 6B 6C 6D 6E 6F"  
SysexLoBitIndex_pt2 = Split (strSysexLoBitIndex_pt2, " ")  
SysexParamChange2 = Array ("F0","3E","0E","00","20","00","01","00","00","F7")  
  
  
For k = 0 to 85  
SysexParamChange2 (7)= SysexLoBitIndex_pt2(k)  
SysexParamChange2 (8)= sysex(sysexIndexUnmappedChannel1_pt2(k))  
  
  
SysexParamOutput2 = Join (SysexParamChange2, " ")  
  
mox.SendSysExString SysexParamOutput2  
Mox.sleep(10)  
Next  
  
End If  
End Sub

So this parses a sysex patch dump sent from the xt into corresponding CC and 10 byte sysex param changes into midi-ox, out of midiox through a virtual midi port into ctrlr, where all matching param knobs/sliders update before your eyes. If renoise accepts 10 byte sysex parameter changes this should also work within renoise to serve a similar purpose to what I’d like to achieve with an xrns tool…

finally hell no there is no stepping on any toes on this project - if you’d like to take a stab at working on this as a tool in renoise, by all means please do, there is no ownership in this stuff, and I have hardly begun on building this tool.

mxb · June 10, 2011, 7:42pm

I’ve had a look at what would be involved with this tool and realised that I can pull in a lot of code from other projects I have done. Yay!

By basing the MIDI interfacing code and parameter saving on the AlphaTrack tool and ‘syx’ file parsing on the FileFormats tool I aim to get a working version for testing by the end of this weekend.

I’ve thought about how the interface to this tool will work and have what I believe to be a neat and tidy solution.

From inital glances, MIDI SDS support could also be implemented (send/receive samples to hardware samplers) reasonably easy, so I may do some groundwork on this aspect also.

Stay tuned and I’ll report back later this weekend.

esaruoho · June 10, 2011, 7:44pm

wonnerful news!

mxb · June 11, 2011, 11:54am

Current development status:

Enumerating MIDI inputs/outputs
Automatically connecting on startup (if enabled)
Save / load preferences
Parsing ‘.syx’ files into native format
Renoise menu integration
File browser integration
‘Dumb’ transmitting SysEx
‘Recording’ SysEx
Saving SysEx files

edit: update

Finished!

Please find a version for testing purposes here (I couldn’t attach it to this post for some reason).

I’ve found that there is a simple SysEx request I can send to the AlphaTrack and it will return a SysEx response. This is the only testing I can do. Everything appears to work fine.

The menu interface looks like this:

On Renoise startup (tool startup) it will add menu items for all available MIDI input and outputs. These can be selected from the menu. The currently selected item is indicated with a ‘tick’. Select ‘None’ to disconnect input/output.

If you wish to save default inputs and output click save preferences. Note that they will not be connected on startup (i.e. ‘None’ will be checked, which may appear that it has not worked) unless ‘Automatically connect’ is selected.

Sending of SysEx files (*.syx) can be done from the menu item (with a file prompt) or from selecting them from the file browser (in ‘Instrument’ mode, but they will appear in all modes).

Recording of SysEx is done as follows:

Select ‘Record SysEx’ - it will become checked. This will start recording all SysEx on the selected midi port to memory. Everytime a SysEx message is received it’s size will be displayed in the status bar.
When finished, select ‘Record SysEx’ again - it will become unchecked. If any SysEx data was received it will ask you where to save the data to (as a ‘.syx’ file)

I believe all the above should enable Renoise to ask as a ‘dumb’ SysEx librarian (i.e. you cannot edit SysEx, but just send and receive messages). There is no two-way handshaking taking place yet.

Please get back to me with how this works as I can only do limited testing.

Enjoy!

NB: Note that I’ve named the tool with a ‘com.mxb’ prefix more to indicate myself as the point of contact for any issues rather than to take ownership of the tool. As this tool will not receive any use for me feel free to suggest further improvements.

esaruoho · June 11, 2011, 2:56pm

I’ll try htis with my v. confused midi setup

Well, my first question is: what does Automatically Connect do?

mxb · June 11, 2011, 3:14pm

Sorry if that wasn’t clear.

If ‘Automatically Connect’ is selected when preferences are saved then the tool will automatically try to connect to the saved ‘default’ MIDI input and output on startup.

esaruoho · June 11, 2011, 3:15pm

What I would really like to see here would be a “Request Sysex”

mxb · June 11, 2011, 3:19pm

What do you mean by ‘Request SysEx’? (I don’t know much about how SysEx works).

Presumably it sends a SysEx to a device and expects a response?

esaruoho · June 11, 2011, 3:28pm

Forget everything I said

esaruoho · June 11, 2011, 3:46pm

Ok, hi. I’ve now tested this with “SysEx Dump ALL” in Korg 03R/W. It records a SysEx Librarian-recognizable Korg SysEx, which is the exact same form as if when SysEx Librarian does it.
I’ve used SysEx Librarian to send the script-generated .syx to the 03R/W and it loads, and so does the SySexLibrarian generated one. They’re of identical information content.

I’ll now try with the Digitech Studio Quad4, I’m not even sure if it sends out SysEx but I’m guessing it should.

mxb · June 11, 2011, 3:53pm

I would do, but you deleted your description of what ‘Request SysEx’ was while I logged in!

I think you described it as sending a request command and recording the output. This can be done by starting recording and then sending ‘premade’ syx files.

Is there a SysEx standard that defines some common SysEx messages. This sounds like a bit of an oxymoron to myself.

I’ve found a few here from the General MIDI v2 Spec and I’m aware of the MIDI Sample Dump Standard. Adding these in would be reasonably simple. I don’t think we should go down the road of adding in every possible feature for a large number of external hardware though.

Glad it’s working!

Note that the tool records everything in SysEx format to the file.

Thanks for testing this, but be careful when dumping back to the hardware!

esaruoho · June 11, 2011, 3:55pm

Request SysEx is to say to the outboard gear, please give me a SysEx dump… The rest of the message became irrelevant because I realized that there were no errors in the sysex dump receive in the script.
What I’m going to try next is to try and get some SysEx .syx files, prog per prog, for the 03rw and test it. Digitech Studio Quad4’s Dump worked and was identical again to the SysExLibrarian received one.

mxb · June 11, 2011, 4:28pm

I understand what you are asking with this, but I believe the message different for each piece of hardware. There is no universal ‘dump’ command that I am aware of.

esaruoho · June 11, 2011, 4:34pm

Ok, that’s fine One should know enough of one’s SysEx gear to be able to start a dump.
So far I haven’t been able to find a method of getting any .syx files for the 03rw to work with either the tool or SysexLibrarian, I hope someone else with SysEx gear will step in and test this tool, my next thing would be to test the Alphajuno2 …

mxb · June 11, 2011, 7:24pm

I’ve started a new thread over here on the Tool Announcments forum to handle testing of the code.

I’ve also added MIDI sample transfer but this is completely untested, so use at your own risk!

mSepsis · June 14, 2011, 7:55pm

MXB -
this is quite mind-blowing the efforts you’ve put into this. I haven’t checked this thread in a few days and it looks like you made some explosive progress since then… I’m going to do some testing with this tonight and get back to you with what I find.

kudos and thank you!

esaruoho · April 8, 2024, 12:06am

unfortunately, @mxb , the link no longer works: