[fixed 3.0] Corrupted Sysex

Renoise Tool Development
1

Hi all,

during these final days of 2013, I decided to continue work on my sysex manager tool. However I got stuck on sysex issues quickly. When dumping the buffer from my Virus C HW synth to Renoise the sysex data gets corrupted.
The sysex data received in my LUA script varies in content and sometimes in size. I’m seeing strange random duplications and omissions of 32 byte chunks of sysex data (the max message size for the sysex_callback function).
As a reference I’m using the MIDI Monitor tool (http://www.snoize.com/MIDIMonitor/). The sysex seen in the MIDI Monitor was correct for every buffer dump of my Virus C.

After an hour of debugging and playing around with my script, I decided to switch to MXB’s SysEx Librarian tool for Renoise. When inspecting its output files, I noticed it exhibited the exact same sysex data corruption! Then I tried several different HW midi ports (MOTU midi express 128, the one on my Novation controller, even the one on my MOTU 828 mk3 soundcard): same corruption issue for all ports. I’ve tried both Renoise 2.81 and Renoise 3.0 beta4. Both have the same problem.

I’m running OSX 10.9.1.

Does anybody have the same problems or perhaps a clue of what’s going on here? The strange thing is that this sysex corruption only occurs in Renoise. None of my other software has any sysex issues.

Hope somebody can help me solve this.

A musically fulfilling 2014 to all of you!!

Cheers,

Erik

2

Spider sense detected a mention of one of my tools…

Do you have a hex dump of what MIDImonitor and Renoise receives? (My tool just writes the raw data, so you can hex dump the file). SysEx messages start with 0xF0 and end with 0xF7, I wonder if there are actual 0xF7’s in the data or there are more than one messages and Renoise is getting confused?

3

Hi mxb, thanks for the quick response :)/>
I’ve made three consecutive buffer dumps of the exact same patch from my Virus C to your tool. Two of the three dumps contain duplications. See the hex dumps below for the details (deviations are marked in red).

I’ve attached the original files as well.

Thanks for helping out!

MIDI Monitor reference

F0 00 20 33 01 00 10 00 7F 07 00 03 2F 01 05 00
7F 06 00 40 7F 00 00 00 00 00 00 0C 1C 40 60 00
00 18 40 16 4B 00 40 40 60 00 7F 32 00 40 00 00
40 14 19 00 00 00 6E 5E 40 00 00 00 01 00 02 00
00 7F 40 00 00 5A 37 22 16 00 00 00 70 02 7F 00
00 00 00 4A 40 40 40 40 32 02 7F 01 16 00 01 40
40 40 40 40 50 01 4C 00 00 00 00 30 7F 40 00 01
00 01 00 00 00 70 50 01 00 02 3C 0B 5A 32 01 12
2B 00 00 00 01 00 00 00 00 00 00 00 00 00 40 00
5E 01 00 00 01 00 00 00 00 39 04 00 00 00 00 7F
00 00 01 42 3E 01 00 01 01 01 24 00 01 00 00 00
00 00 1B 50 34 00 2B 55 40 40 40 40 00 00 40 40
40 40 40 00 40 40 40 5E 1B 14 10 00 1A 2A 36 32
65 13 27 2C 29 22 00 40 00 40 14 00 00 02 00 00
00 55 40 7F 00 40 47 33 40 40 40 2A 00 00 00 00
03 00 40 03 00 40 03 00 40 48 61 72 70 73 69 65
20 48 53 00 04 07 01 00 7F 3E F7

SysEx Librarian

dump1:

F0 00 20 33 01 00 10 00 7F 07 00 03 2F 01 05 00
7F 06 00 40 7F 00 00 00 00 00 00 0C 1C 40 60 00
00 18 40 16 4B 00 40 40 60 00 7F 32 00 40 00 00
40 14 19 00 00 00 6E 5E 40 00 00 00 01 00 02 00
00 7F 40 00 00 5A 37 22 16 00 00 00 70 02 7F 00
00 00 00 4A 40 40 40 40 32 02 7F 01 16 00 01 40
40 40 40 40 50 01 4C 00 00 00 00 30 7F 40 00 01
00 01 00 00 00 70 50 01 00 02 3C 0B 5A 32 01 12
2B 00 00 00 01 00 00 00 00 00 00 00 00 00 40 00
5E 01 00 00 01 00 00 00 00 39 04 00 00 00 00 7F
00 00 01 42 3E 01 00 01 01 01 24 00 01 00 00 00
00 00 1B 50 34 00 2B 55 40 40 40 40 00 00 40 40
40 40 40 00 40 40 40 5E 1B 14 10 00 1A 2A 36 32
65 13 27 2C 29 22 00 40 00 40 14 00 00 02 00 00
20 48 53 00 04 07 01 00 7F 3E F7 2A 00 00 00 00
03 00 40 03 00 40 03 00 40 48 61 72 70 73 69 65
20 48 53 00 04 07 01 00 7F 3E F7

dump2:

F0 00 20 33 01 00 10 00 7F 07 00 03 2F 01 05 00
7F 06 00 40 7F 00 00 00 00 00 00 0C 1C 40 60 00
00 18 40 16 4B 00 40 40 60 00 7F 32 00 40 00 00
40 14 19 00 00 00 6E 5E 40 00 00 00 01 00 02 00
00 7F 40 00 00 5A 37 22 16 00 00 00 70 02 7F 00
00 00 00 4A 40 40 40 40 32 02 7F 01 16 00 01 40
40 40 40 40 50 01 4C 00 00 00 00 30 7F 40 00 01
00 01 00 00 00 70 50 01 00 02 3C 0B 5A 32 01 12
2B 00 00 00 01 00 00 00 00 00 00 00 00 00 40 00
5E 01 00 00 01 00 00 00 00 39 04 00 00 00 00 7F
00 00 01 42 3E 01 00 01 01 01 24 00 01 00 00 00
00 00 1B 50 34 00 2B 55 40 40 40 40 00 00 40 40
40 40 40 00 40 40 40 5E 1B 14 10 00 1A 2A 36 32
65 13 27 2C 29 22 00 40 00 40 14 00 00 02 00 00
00 55 40 7F 00 40 47 33 40 40 40 2A 00 00 00 00
03 00 40 03 00 40 03 00 40 48 61 72 70 73 69 65
20 48 53 00 04 07 01 00 7F 3E F7

dump3:

F0 00 20 33 01 00 10 00 7F 07 00 03 2F 01 05 00
7F 06 00 40 7F 00 00 00 00 00 00 0C 1C 40 60 00
00 7F 40 00 00 5A 37 22 16 00 00 00 70 02 7F 00
00 00 00 4A 40 40 40 40 32 02 7F 01 16 00 01 40
00 7F 40 00 00 5A 37 22 16 00 00 00 70 02 7F 00
00 00 00 4A 40 40 40 40 32 02 7F 01 16 00 01 40
40 40 40 40 50 01 4C 00 00 00 00 30 7F 40 00 01
00 01 00 00 00 70 50 01 00 02 3C 0B 5A 32 01 12
00 00 01 42 3E 01 00 01 01 01 24 00 01 00 00 00
00 00 1B 50 34 00 2B 55 40 40 40 40 00 00 40 40
00 00 01 42 3E 01 00 01 01 01 24 00 01 00 00 00
00 00 1B 50 34 00 2B 55 40 40 40 40 00 00 40 40
00 55 40 7F 00 40 47 33 40 40 40 2A 00 00 00 00
03 00 40 03 00 40 03 00 40 48 61 72 70 73 69 65
00 55 40 7F 00 40 47 33 40 40 40 2A 00 00 00 00
03 00 40 03 00 40 03 00 40 48 61 72 70 73 69 65
20 48 53 00 04 07 01 00 7F 3E F7

4

Since Renoise 3.0 has a nice MIDI panel. I’ve written down the sysex data as monitored by this panel. For completeness I’ve also added a screenshot of the MIDI panel.

The last 3 lines of the Renoise MIDI panel data are particularly interesting. As you can see, the “” sysex data actually has been received by a buffer somewhere.
But before Renoise has read this buffer and fed this data to the MIDI panel, the buffer has already been overwritten by the last 11 bytes of the sysex dump.

MIDI Monitor reference (verified to be correct)

F0 00 20 33 01 00 10 00 7F 07 00 03 2F 01 05 00
7F 06 00 40 7F 00 00 00 00 00 00 0C 1C 40 60 00
00 18 40 16 4B 00 40 40 60 00 7F 32 00 40 00 00
40 14 19 00 00 00 6E 5E 40 00 00 00 01 00 02 00
00 7F 40 00 00 5A 37 22 16 00 00 00 70 02 7F 00
00 00 00 4A 40 40 40 40 32 02 7F 01 16 00 01 40
40 40 40 40 50 01 4C 00 00 00 00 30 7F 40 00 01
00 01 00 00 00 70 50 01 00 02 3C 0B 5A 32 01 12
2B 00 00 00 01 00 00 00 00 00 00 00 00 00 40 00
5E 01 00 00 01 00 00 00 00 39 04 00 00 00 00 7F
00 00 01 42 3E 01 00 01 01 01 24 00 01 00 00 00
00 00 1B 50 34 00 2B 55 40 40 40 40 00 00 40 40
40 40 40 00 40 40 40 5E 1B 14 10 00 1A 2A 36 32
65 13 27 2C 29 22 00 40 00 40 14 00 00 02 00 00
00 55 40 7F 00 40 47 33 40 40 40 2A 00 00 00 00
03 00 40 03 00 40 03 00 40 48 61 72 70 73 69 65
20 48 53 00 04 07 01 00 7F 3E F7

Renoise 3.0b3 MIDI panel:

F0 00 20 33 01 00 10 00 7F 07 00 03 2F 01 05 00
7F 06 00 40 7F 00 00 00 00 00 00 0C 1C 40 60 00
00 18 40 16 4B 00 40 40 60 00 7F 32 00 40 00 00
40 14 19 00 00 00 6E 5E 40 00 00 00 01 00 02 00
2B 00 00 00 01 00 00 00 00 00 00 00 00 00 40 00
5E 01 00 00 01 00 00 00 00 39 04 00 00 00 00 7F
2B 00 00 00 01 00 00 00 00 00 00 00 00 00 40 00
5E 01 00 00 01 00 00 00 00 39 04 00 00 00 00 7F
2B 00 00 00 01 00 00 00 00 00 00 00 00 00 40 00
5E 01 00 00 01 00 00 00 00 39 04 00 00 00 00 7F
40 40 40 00 40 40 40 5E 1B 14 10 00 1A 2A 36 32
65 13 27 2C 29 22 00 40 00 40 14 00 00 02 00 00
40 40 40 00 40 40 40 5E 1B 14 10 00 1A 2A 36 32
65 13 27 2C 29 22 00 40 00 40 14 00 00 02 00 00

20 48 53 00 04 07 01 00 7F 3E F7 2A 00 00 00 00
03 00 40 03 00 40 03 00 40 48 61 72 70 73 69 65

5

…could it be
that i can finally
save and load my virus sysex-patches
within renoise edges?

…when is start to rhyme,
then i’m quiet frustrated.

Hope-is-a-lack-of-information-greetings
Hardock

6

I will have a go with some of my synths and get back to you

7

It looks like an offset problem in the buffer, it seems the lines that are incorrect are duplicated, so somehow it may be that the offset of the last picked buffer-data is incorrect and some previously fetched data is being refetched again but at the size of the last block of the buffer.
The parts i made bold are an exact copy of what was fetched earlier in red.

8

This got fixed during the 3.0 betas.

9

Super!

Now a sysex dump receive, store and send device could be possible?

mbx had some doubts about storing sysex dumps in a renoise song:

https://forum.renoise.com/t/new-tool-2-7-2-8-sysex-librarian/32866