Send And Receive Sysex Dumps To/From Outboard Gear

Renoise Tool Development
1

So I’m taking a break from my development work on ctrlr panels until ctrlr v5 development slows down a little and the linux and osx versions catch up a little bit. . One month of working in windows again was about all I could take.

I’m moving on to attempt to get a working prototype of an xrns tool which can receive, record and playback sysex patch dumps. I’m going to start… you guessed it… with the microwave XT.

I’ve done a good bit of reading up on Lua, finally enabled the scripting terminal inside renoise and am curious to begin with if anybody who has thought about this or has any ideas how to go about tackling this might have some any helpful suggestions since I’v been told here that the renoise scripting API has everything it needs to do what I’m talking about.

I’m inspired by the work Joel Johansson started with a parser for cakewalk instrument definition files here, not to mention the robustness of the code library and integration into renoise. I love the fact that not one person has posted a reply to taktik’s “thread” regarding reporting any crashes of renoise. That’s some solid testament right there!

So I’m looking for suggestions as to which snippets and ingredients that are already there might anybody recommend I start working with rather than chasing down a rabbit hole (my favorite phrase as of late)

I want this thread to be open… this is not my project, its everybody’s.

2

I could google :slight_smile: , but just out of curiosity as I don’t own any midi hardware and am strictly vst(i), what would this sysex dump enable you that can’t be done now? I keep hearing the term, but have no idea about the functionality.

cheers.

3

I can test it with Roland Alpha-Juno2 and Korg 03R/W, both of which are calable of sysex dumps. thx!

4

A “sysex dump” is a transfer of data that contains all of the current parameter settings for a MIDI synthesizer.

This data can be sent from the synth (after youve tweaked the sound to your liking) to the DAW and recorded, usually placed at the header or first measure of a song, before any notes are played.

When the DAW then plays back the song from that first measure, the sysex data is sent to the synth, setting all of the current parameters to the synth.

When a DAW supports receiving, recording and playing back sysex dumps it allows the sounds to be saved with the song, kind of like how VSTs have done without anybody really thinking about it…

Throughout time, vsts have caused a slight amnesia and (imho) misplaced attention away from the true workhorses of electronic music - MIDI synthesizers.

Imagine if you weren’t able to save your vst sounds with your song’s project file. That’s what its like these days with most daws who have let MIDI synths fall to the wayside in the equation of electronic music production. It’s a massive headache and hampers creativity when needing to manage all these sounds and ensure that the sound you’re saving over on the synth isn’t used in some other song you wrote 3 months ago. don’t forget many very powerful very popular synths (Nord Lead 2 for example) don’t save patches with a name, but instead number each patch (aka instrument, program or sound)

5

Send and receive sysex messages via Renoise API:
http://code.google.c…ippets/Midi.lua

Using the “Document API” to create persistent settings (in this case, to build a library of sysex dumps)
http://code.google.c…ocument.API.lua

I think the biggest challenge with a project like this is not the MIDI communication itself, but rather to maintain the persistent documents/settings. Unfortunately, the Document API is kind of tricky to work with - but then, as you just pointed out you’re not alone with this project

Also needed is a good way of matching a particular song with particular preset(s). BeatSlaughter has done some groundwork with his Mixer Utilities tool, which will simply save a special file next to the song telling the tool that “this particular song has some customized settings”. Perhaps the same approach could be applied to this tool?

6

I ported a generic Midi class:

It’s not for real time streams, but it could save you some time if you are reading/writing .MID files.

7

Thanks for the info, that sucks indeed if you have a lot of midi gear stuff and have to manually adjust it to correspond to a particular song.

Long time ago I had a Roland Alpha Juno 1 and I remember having Renoise remembering program (preset) changes and midi automation in the pattern, but I guess this is something else than what you are looking for?

8

There are actual SYSEX-based tools for editing the sounds of the Roland Alpha Juno1, for instance. There are at least two such environments for Logic, and standalone software also. To be able to have something like this, or even just sysex dumps etc, safely in Renoise, will assist in at least retaining patches.

9

Thank you very much Conner, and thank you Danoise for the leads. I’ve looked up to both of you many times before on the forums, so I appreciate you both taking some time to give some input here.

could you elaborate please on the contextual implications of this class being “not for real time streams”? I’m assuming this implys it’s not a class that can be dealt with in anyway within renoie if a song or pattern is playing?

Thanks again

10

damn i had a juno-6 that didn’t have MIDI. I guess Roland was dyslexic. (Note to all: never ever sell any gear… unless it’s a sampler you sold after renoise 2.7 beta hit :)

so, Jonas… so say you happen to save over your program(preset) on your Juno 1 with a new sound while you’re creating a new song. Then you go back to the song you made using this same preset before you saved over the program(preset)… … …

11

It just means it’s for MIDI files.

Otherwise, MIDI I/O can be done with Renoise quite easily. See Danoise’s link and the corresponding doc, here:

12

mSepis,

Good luck with the development of this tool. I’m sure you’ll end up learning much more than you think you will at the start!

If I may suggest a basic procedure for recording/saving SysEx:

This could open a GUI dialog that asks the user which MIDI device name / channel to send it on.

This may be an alternative method that would allow a user to have a normal folder structure to organise SysEx files. I’ve done a fair amount of work with file-browser interfacing for the File Formats tool, so I can assist with this integration if you wish. (It’s fairly easy to integrate once the tool functionality is there - I would suggest this is added towards the end).

Good luck!

13

mSepsis / anyone else,

Has anyone done any work on this?

I’ve got a quiet weekend coming up and wouldn’t mind spending some time working on this.

I don’t have any external hardware to test with but I can at least make some of the foundations. I’d be aiming to do something like the following:

  • Enumerating / Binding to midi ports
  • Send / Receive sysex routines
  • Saving preferences (last used midi ports etc.)
  • Loading / Saving .syx format files
  • Integration into the native file browser

However, I don’t want to tread on anyone elses toes or reduplicate others work. Thus, this post to let everyone else know my current intentions. :)

14

I don’t think anyone’s necessarily doing anything.
I did a really minor thing, a currently_selected_instrument’s midi_program change:

function midiprogram(change)  
local midi=renoise.song().selected_instrument.midi_output_properties  
local currentprg=midi.program  
 currentprg = math.max(1, math.min(128, currentprg + change))  
 rprint (currentprg)  
 print ("yo")  
renoise.song().selected_instrument.midi_output_properties.program = currentprg  
renoise.song().transport:panic()  
end  
  
--renoise.song().selected_instrument.midi_output_properties.program =   
-->>> oprint (renoise.song().selected_instrument.midi_output_properties)  
--class: InstrumentMidiOutputProperties  
 --properties:  
 --bank  
 --bank_observable  
 --channel  
 --channel_observable  
 --delay  
 --delay_observable  
 --device_name  
 --device_name_observable  
 --duration  
 --duration_observable  
 --instrument_type  
 --instrument_type_observable  
 --program  
 --program_observable  
 --transpose  
 --transpose_observable  
  
renoise.tool():add_keybinding { name = "Global:Impulse:Next/Prev Midi Program +1",   
invoke = function() midiprogram(1)  
end }  
renoise.tool():add_keybinding { name = "Global:Impulse:Next/Prev Midi Program -1",   
invoke = function() midiprogram(-1)  
end }

That’s all I know about

15

I have a vbscript that when run in midiox converts the microwave’s sysex dump to controller changes… somewhat related project to this, but no besides buying a few Lua books and reading up and experimenting I haven’t made any real progress to report on building this tool. I’m happy to see there has been some interest and it’s not just me wishing for this in renoise…

here is the vbscript if anyone is interested at all, credit goes to “fozzie” over on the ctrlr forums. I designed, built and coded the ctrlr, he wrote this script.

  
' Create object  
Set mox = WScript.CreateObject("MIDIOX.MOXScript.1", "OnTrigger_")  
  
mox.DivertMidiInput = 0 'no input diversion  
mox.FireMidiInput = 1 ' Start processing MIDI  
MsgBox "Press OK to end MIDI loop" ' Will not return until after [OK]  
mox.FireMidiInput = 0 ' All done processing  
mox.DivertMidiInput = 0  
  
mox.ShutdownAtEnd = False  
  
Set mox = nothing ' Cleanup  
  
  
Sub OnTrigger_SysExInput (strSysex)  
If len(strSysEx) > 100 Then  
sysex = Split(strSysEx)  
  
port = -1  
  
'MsgBox strSysEx  
  
'Conversion of all relevant Sysex parameters to implemented CC messages on channel 1  
strsysexIndexCC = "8 9 10 12 13 14 19 20 21 23 24 25 26 32 33 34 35 36 37 38 43 44 45 46 47 48 49 54 55 56 57 69 70 71 72 73 74 80 81 82 84 86 87 89 91 94 95 96 97 99 100 101 102 103 104 105 106 107 120 121 122 123 124 126 127 128 129 130 156 157 158 159 160 161 162 163 164 166 167 168 169 170 171 173 174 175 176 177 178 179"  
sysexIndexCC = Split (strsysexIndexCC, " ")  
strccIndex = "33 34 35 36 37 13 38 39 40 41 42 43 44 70 71 72 73 74 75 76 77 78 79 80 81 82 83 45 46 47 48 50 56 54 51 52 53 60 61 62 57 58 55 12 10 65 22 23 5 102 105 104 103 107 106 108 109 110 14 15 16 17 29 18 19 20 21 31 85 86 87 88 89 90 91 92 93 24 25 30 112 113 114 26 28 27 115 116 117 118"  
ccIndex = Split (strccIndex, " ")  
For i = 0 to 89  
mox.OutputMidiMsg port, 176, ccIndex(i), cint("&H" & sysex(sysexIndexCC(i)))  
Mox.sleep(10)  
Next  
  
'part two of conversion; the parameters that are not coupled to a CC will be transformed into separate sysex parameter change messages (part 1 for parameters 0-127  
  
strsysexIndexUnmappedChannel1 = "58 60 61 62 64 77 83 88 90 92 93 108 109 110 111 112 115 116 117 119 132 133 134"  
sysexIndexUnmappedChannel1 = Split (strsysexIndexUnmappedChannel1, " ")  
strSysexLoBitIndex = "33 35 36 37 39 46 4C 51 53 55 56 65 66 67 68 69 6C 6D 6E 70 7D 7E 7F"  
SysexLoBitIndex = Split (strSysexLoBitIndex, " ")  
SysexParamChange = Array ("F0","3E","0E","00","20","00","00","00","00","F7")  
  
  
For j = 0 to 22  
SysexParamChange (7)= SysexLoBitIndex(j)  
SysexParamChange (8)= sysex(sysexIndexUnmappedChannel1(j))  
  
SysexParamOutput = Join (SysexParamChange, " ")  
  
mox.SendSysExString SysexParamOutput  
Mox.sleep(10)  
Next  
  
'part three of conversion; the parameters that are not coupled to a CC will be transformed into separate sysex parameter change messages (part 2 for parameters 128-256  
  
  
strsysexIndexUnmappedChannel1_pt2 = "135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246"  
sysexIndexUnmappedChannel1_pt2 = Split (strsysexIndexUnmappedChannel1_pt2, " ")  
strSysexLoBitIndex_pt2 = "00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F 10 11 12 13 14 2F 30 31 32 33 34 35 36 37 38 39 3A 3B 3C 3D 3E 3F 40 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 51 52 53 54 55 56 57 58 59 5A 5B 5C 5D 5E 5F 60 61 62 63 64 65 66 67 68 69 6A 6B 6C 6D 6E 6F"  
SysexLoBitIndex_pt2 = Split (strSysexLoBitIndex_pt2, " ")  
SysexParamChange2 = Array ("F0","3E","0E","00","20","00","01","00","00","F7")  
  
  
For k = 0 to 85  
SysexParamChange2 (7)= SysexLoBitIndex_pt2(k)  
SysexParamChange2 (8)= sysex(sysexIndexUnmappedChannel1_pt2(k))  
  
  
SysexParamOutput2 = Join (SysexParamChange2, " ")  
  
mox.SendSysExString SysexParamOutput2  
Mox.sleep(10)  
Next  
  
End If  
End Sub

So this parses a sysex patch dump sent from the xt into corresponding CC and 10 byte sysex param changes into midi-ox, out of midiox through a virtual midi port into ctrlr, where all matching param knobs/sliders update before your eyes. If renoise accepts 10 byte sysex parameter changes this should also work within renoise to serve a similar purpose to what I’d like to achieve with an xrns tool…

finally hell no there is no stepping on any toes on this project - if you’d like to take a stab at working on this as a tool in renoise, by all means please do, there is no ownership in this stuff, and I have hardly begun on building this tool.

16

I’ve had a look at what would be involved with this tool and realised that I can pull in a lot of code from other projects I have done. Yay!

By basing the MIDI interfacing code and parameter saving on the AlphaTrack tool and ‘syx’ file parsing on the FileFormats tool I aim to get a working version for testing by the end of this weekend.

I’ve thought about how the interface to this tool will work and have what I believe to be a neat and tidy solution.

From inital glances, MIDI SDS support could also be implemented (send/receive samples to hardware samplers) reasonably easy, so I may do some groundwork on this aspect also.

Stay tuned and I’ll report back later this weekend.

1 Like
17

wonnerful news! :walkman:

18

Current development status:

  • Enumerating MIDI inputs/outputs
  • Automatically connecting on startup (if enabled)
  • Save / load preferences
  • Parsing ‘.syx’ files into native format
  • Renoise menu integration
  • File browser integration
  • ‘Dumb’ transmitting SysEx
  • ‘Recording’ SysEx
  • Saving SysEx files

edit: update

Finished! :)

Please find a version for testing purposes here (I couldn’t attach it to this post for some reason).

I’ve found that there is a simple SysEx request I can send to the AlphaTrack and it will return a SysEx response. This is the only testing I can do. Everything appears to work fine.

The menu interface looks like this:

On Renoise startup (tool startup) it will add menu items for all available MIDI input and outputs. These can be selected from the menu. The currently selected item is indicated with a ‘tick’. Select ‘None’ to disconnect input/output.

If you wish to save default inputs and output click save preferences. Note that they will not be connected on startup (i.e. ‘None’ will be checked, which may appear that it has not worked) unless ‘Automatically connect’ is selected.

Sending of SysEx files (*.syx) can be done from the menu item (with a file prompt) or from selecting them from the file browser (in ‘Instrument’ mode, but they will appear in all modes).

Recording of SysEx is done as follows:

  • Select ‘Record SysEx’ - it will become checked. This will start recording all SysEx on the selected midi port to memory. Everytime a SysEx message is received it’s size will be displayed in the status bar.
  • When finished, select ‘Record SysEx’ again - it will become unchecked. If any SysEx data was received it will ask you where to save the data to (as a ‘.syx’ file)

I believe all the above should enable Renoise to ask as a ‘dumb’ SysEx librarian (i.e. you cannot edit SysEx, but just send and receive messages). There is no two-way handshaking taking place yet.

Please get back to me with how this works as I can only do limited testing.

Enjoy!

NB: Note that I’ve named the tool with a ‘com.mxb’ prefix more to indicate myself as the point of contact for any issues rather than to take ownership of the tool. As this tool will not receive any use for me feel free to suggest further improvements.

1 Like
19

I’ll try htis with my v. confused midi setup ;)

Well, my first question is: what does Automatically Connect do?

20

Sorry if that wasn’t clear.

If ‘Automatically Connect’ is selected when preferences are saved then the tool will automatically try to connect to the saved ‘default’ MIDI input and output on startup.