Best Ways To Detect Changes To A Pattern?

Renoise Tool Development
1

I’ve looked at the API docs, and it doesn’t seem like we have a general method for determining if a pattern has changed?

Maybe there’s a way to do this that doesn’t involve having a complete “cached” copy of the pattern, and then doing a note-by-note comparison? Because that was the first idea I had, but it also sounds like a relatively heavy thing to do, CPU-wise…

There’s two thing I want to use this for: first of all, I’d like to improve the Duplex StepSequencer by always having it’s sequence in sync with the pattern (when you edit a pattern in Renoise, currently the sequencer doesn’t update it’s display accordingly). And then I have an idea about being able to compare matrix slots and update them, based on a memorized version (sort of a poor man’s clip implementation).

2

This is the kind of thing where in the past I have asked if checking the Undo Buffer would be enough. Poll the Undo Buffer every X miliseconds for any changes of data in relevant pattern? As there are only ever likely to be few new Undo points would hope it to less intensive… (Although don’t even know if it’s allowed by the API.)

3

The following code will compare any two tracks, and look for note/effect commands, or both.
It also has a upper/lower line number, to compare a specific part of the pattern.

Execution time is OK, but I’m sure it could be improved. And it doesn’t tell us if anything has changed in the first place, we’d still need some kind of notifier for that.

-- tracks_are_identical() - compare tracks  
--  
-- note: the "visible_only" will enforce a stricter comparison between   
-- tracks when set to true. For instance, trying to compare two   
-- patterns with a "line_from" parameter that exceed the track lines   
-- in either track will return nil and print a warning  
-- also, if the tracks are identical except from the number of columns,  
-- comparing with the "visible_only" option will return false -   
-- no matter what the contents are  
--   
-- @track1, track2 (renoise.PatternTrack) tracks to compare  
-- @visible_only (boolean) ignore hidden columns  
-- @include_notes (boolean) include note-columns  
-- @include_effects (boolean) include effect-columns  
-- @line_from,line_to (integer) compare selected lines  
-- @return (boolean) true if matched, nil if "conditions not met"  
  
function tracks_are_identical(patt1_idx,track1_idx,patt2_idx,track2_idx,visible_only,include_notes,include_effects,line_from,line_to)  
  
 print (string.format("Comparing pattern %i, track %i with pattern %i, track %i...limited to visible columns: %s, include_notes: %s, include_effects: %s", patt1_idx,track1_idx,patt2_idx,track2_idx,(visible_only and "yes" or "no"),(include_notes and "yes" or "no"),(include_effects and "yes" or "no")))  
  
 local rslt = true  
  
 -- quick test for identical indices  
 if (patt1_idx==patt2_idx) and (track1_idx==track2_idx) then  
 return true  
 end  
  
 -- check pre-conditions  
 if (not include_notes) and (not include_effects) then  
 print("No basis for comparison - you need to compare notes and/or effects")  
 return  
 end  
  
 local patt1 = renoise.song().patterns[patt1_idx]  
 local patt2 = renoise.song().patterns[patt2_idx]  
 if (not patt1) or (not patt2) then  
 print("One of the specified patterns could not be located")  
 return  
 end  
  
 local track1 = patt1.tracks[track1_idx]  
 local track2 = patt2.tracks[track2_idx]  
 if (not patt1) or (not patt2) then  
 print("One of the specified tracks could not be located")  
 return  
 end  
  
 if visible_only and line_from then   
 if (patt1.number_of_lines<line_from then></line_from> print("Invalid range specified: line_from higher than visible lines in pattern1")  
 return   
 elseif (patt2.number_of_lines<line_from then></line_from> print("Invalid range specified: line_from higher than visible lines in pattern2")  
 return   
 end  
 end  
  
 -- pre-check: one of the tracks are empty?  
 rslt = (track1.is_empty == track2.is_empty)  
 if (not rslt) and (include_notes) and (include_effects) then  
 -- since we care about every type of content, it's safe to  
 -- say that the tracks are different (it's not enough to  
 -- detect that one track is empty if the other track contained  
 -- a note, and we were only interested in effects)  
 return false  
 end  
  
 -- compare notes  
 if (include_notes) then  
 rslt = __compare_columns("note",patt1_idx,track1_idx,patt2_idx,track2_idx,visible_only,line_from,line_to)  
 end  
  
 -- compare effects  
 if (rslt) and (include_effects) then  
 rslt = __compare_columns("effect",patt1_idx,track1_idx,patt2_idx,track2_idx,visible_only,line_from,line_to)  
 end  
  
 return rslt  
  
end  
  
----------------------------------------------------------------------  
  
-- compare_columns() - this method should be as fast as possible   
-- @type (string) "note" or "effect"  
-- @patt1_idx/patt2_idx (integer)  
-- @track1_idx/track2_idx (integer)  
-- @visible_only (boolean)  
-- @line_from,line_to (integer)   
  
function __compare_columns(type,patt1_idx,track1_idx,patt2_idx,track2_idx,visible_only,line_from,line_to)  
  
 local counter = 1  
 local cached = table.create()  
 local pattern_iter = renoise.song().pattern_iterator  
 local iter = (type=="note") and pattern_iter:note_columns_in_pattern_track(patt1_idx,track1_idx,visible_only)  
 or pattern_iter:effect_columns_in_pattern_track(patt1_idx,track1_idx,visible_only)  
 local matched = nil  
 -- match relevant lines for comparison  
 for pos,column in iter do  
 matched = true  
 if (line_from) then  
 if (pos.line<line_from then></line_from> matched = false  
 elseif (pos.line>line_to) then  
 break  
 end  
 end   
 if (matched) then   
 table.insert(cached,pos.column..":"..column:__tostring())  
 end  
 end  
 iter = (type=="note") and pattern_iter:note_columns_in_pattern_track(patt2_idx,track2_idx,visible_only)  
 or pattern_iter:effect_columns_in_pattern_track(patt2_idx,track2_idx,visible_only)  
 -- compare lines   
 for pos,column in iter do   
 matched = true  
 if (line_from) then  
 if (pos.line<line_from then></line_from> matched = false  
 elseif (pos.line>line_to) then  
 break  
 end  
 end   
 if (matched) then  
 if (pos.column..":"..column:__tostring() ~= cached[counter]) then   
 return false  
 end  
 counter = counter+1  
 end  
 end   
  
 return true  
  
end  
  
----------------------------------------------------------------------  
  
-- now test the compare function (with timer)  
  
local before = os.clock()  
local patt1_idx = 1  
local track1_idx = 1  
local patt2_idx = 2  
local track2_idx = 1  
local visible_only = false  
local include_notes = false  
local include_effects = true  
local line_from = 8  
local line_to = 16  
--local line_from = nil  
--local line_to = nil  
  
local rslt = tracks_are_identical(patt1_idx,track1_idx,patt2_idx,track2_idx,visible_only,include_notes,include_effects,line_from,line_to)  
print("Result:",rslt," - execution time (seconds):",(os.clock()-before))
4

If its just about comparing existing patterns/tracks, then Renoise can do this better internally and we only have to publish comparison operators, but this doesn’t really allow to track for pattern content changes, cause you need some “old” pattern data to compare then?

Thought about notifiers for pattern data as well, but the problem with those is that they are causing a LOT of overhead. They would be called for every single change within patterns, in every script which has one, like for example hundreds of times when simply clearing a pattern. I do understand though that workaround around missing pattern data notifiers simply is quite awful right now.

5

If the idea is to have a notifier that detects changes to tracks, then it’s up to the script author to act upon it - the notifier itself does not create overhead, or?

If so, the potential performance of scripts isn’t really an issue IMHO. Done right, this should all be pretty light-weight stuff.
Just to give you an example, the step sequencer could detect changes to the selected pattern, but only refresh it’s display when the change has occurred somewhere between line X and Y (the part of the pattern you’re actively editing).

Hmm…selected_pattern_changed_observable is probably the most manageable option, have to think more about it.

6

Will try this out to see if its relevant.

Btw: lines, and columns already have a comparison operator, so you can already do:

if (some_line == other_line) then
– same line content

if (some_note_column == other_note_column) then
– same note_column content

if (some_effect_column == other_note_effect_column) then
– same effect_column content

I’ll publish comparison operators for patterns, pattern tracks and automation as well.

7

Wow, I must’ve overlooked that! Currently, comparing two tracks with 64 lines, three columns still takes a fraction of a second on my PC. Curious just how much better performance I can squeeze from the compare function then

And 1000x thanks for considering something like this so late in the release. The change notifier is by far the most important IMHO, I was talking about the “selected_pattern_changed_observable” as a manageable option because we as script authors would not need to do a lot of housekeeping then to track changes to the active pattern , it would always follow the pattern editor’s focus (it would not track changes to patterns that aren’t selected - meaning: another tool modifying some other part of the song - but I don’t think that’s likely to be a big issue)

8

We would need change notifiers that operate in four different scopes: pattern, pattern_column, pattern_column_row, and pattern_range (for block operations).

That should take care of the notification overhead - for changes affecting an entire pattern, fire one event; for changes affecting a column, just one event. Block cut/paste/delete can fire as one pattern_range event with pattern number and row/column start and end.

Events need to fire after user changes only, not after changes generated via the API, since this would cause infinite recursion for scripts that respond to changes by making changes themselves. Alternatively, we’d need an option to temporarily suspend event generation while making changes in a script.

I’ve been thinking about writing the much-talked-about “network tracking” plugin, but by the sound of it, LUA may be too slow? I’ve used a lot of languages, but never LUA, so I don’t know much about it. Someone mentioned it took a fraction of a second just to compare two patterns - makes me wonder if it’s even feasible to do the network tracking feature with the scripting API at all?

Either way, it sounds like I would need these more granular change events to implement this efficiently. Would you take that under consideration for a future release?

9

mindplay: this topic is out of date. There are notifiers for pattern data already. See https://code.google.com/p/xrnx/source/browse/trunk/Documentation/Renoise.Song.API.lua#1390 please

I also can’t guess what exactly will be the bottleneck in this case. The pattern line notifiers in Lua, in Renoise, the network stuff. I think no one here can.
As soon as we know what exactly needs more optimization we can do this later on?

Also if you do pattern data comparison not in Lua, but let Renoise do them ->

  
-- Compares all tracks and lines, including automation.  
==(Pattern object, Pattern object) -> [boolean]  
~=(Pattern object, Pattern object) -> [boolean]  
-- see also pattern track/line and other related operators

you can avoid !any! Lua overhead. On the other hand, Lua is blazing fast. That was one of the main reasons we decided to go for Lua.