What is the fastest way to iterate through notes in a pattern?

Renoise Tool Development
1

i have a script that iterates through “note ons” in a pattern and runs some logic on them - i’m using the pattern iterator right now as below:

for pos, col in renoise.song().pattern_iterator:note_columns_in_pattern(song.selected_pattern_index, false) do
if (not col.is_empty) and (col.note_value ~= 120) then
  -- do stuff with col.note_value
end 
end

i’ve noticed that if the script runs while the song is playing, the UI stutters (i assume because the script takes some time to process). is there a faster way to do this?

sh

2

Code seems legit. But consider if you need to process hidden columns (the second argument) ?

Obviously, running through, and applying some function on every note column in a pattern is bound to carry some penalty.

Alternatively (and a lot more complicated to implement), you could split the iteration into smaller tasks which are executed by means of an idle loop notifier.

Then you could use a less taxing iterator likelines_in_range, or examine only a single track at a time, and “level out” the CPU utilization a bit more.

3

thanks!! i will try excluding “hidden” columns, i have a feeling that will be enough to make this workable.

is “hidden” the same thing as “collapsed/expanded”, or does it have another meaning?

sh

4

Every track in Renoise can have 12 note columns, and technically they always there.

So, when you choose to “add” a note column, you are in fact just “un-hiding” it.

This is also why hiding note columns does not destroy the note data…

5

thanks for the explanation.

encountering a pretty weird behaviour - it seems like setting the “visible_only” flag to true, it’s actually making the problem worse (just counting roughly it increases the lagtime from one beat to 4 or 5 beats). am i missing something?

6

i removed all my code and just put a print statement in there, and i’m getting the same results - in theory this code should run faster than my initial version?

for pos, col in renoise.song().pattern_iterator:note_columns_in_pattern(song.selected_pattern_index, true) do
    -- ignore empty columns, and note_off (120) 
    if (not col.is_empty) and (col.note_value ~= 120) then       
      print("test")                             
    end           
  end
7

From 1 to 4/5 beats lag time, that does sound weird indeed.

Obviously, you would only get a performance increase when your tracks are not using all 12 note columns :wink:

8

The fastest way will probably be if you don’t use the pattern iterator but make the code yourself, only what you need. The iterator code is in lua, and it was posted somewhere, can’t find the link though.

“The point of the iterator is to get people quickly into coding in the pattern.”

Search for the code, and you’ll see what you need and what you don’t.

9

Search for the code, and you’ll see what you need and what you don’t.

In this case, the code would search for a needle in a haystack, so you need the fastest possible way to flip through the hay…

Unless you had some idea about where to search, of course. Insider tip?

10

In this case, the code would search for a needle in a haystack, so you need the fastest possible way to flip through the hay…

Unless you had some idea about where to search, of course. Insider tip?

I just suggested that personal code would be written to see if it would be faster. In the scenarios I have used it, it was.

Either way, link is down https://code.google.com/p/xrnx/source/browse/trunk/UnitTests/TestPatternIterators.luaAssuming that link is correct, I don’t have it savedon this computer, I’m sorry.

so can’t quite look at it anymore… :c

11

thanks for the hints - i’ll keep digging away and hopefully find something workable.

i’m using this script to send information about the pattern notes over osc to a visualizer (written in processing) that will create projections - have some tests working and i think it will be a good system for my next tour. thanks again for the help!

sh

12

https://code.google.com/p/xrnx/source/browse/trunk/UnitTests/TestPatternIterators.lua

Might be handy, saved for posterity from google cache

--[[--------------------------------------------------------------------------
TestPatternIterators.lua
--------------------------------------------------------------------------]]--

--[[

The following iter functions are the ones that are defined in
Renoise - exactly as provided below. They are copy & pasted for more
finetuning & testing, cause the internal LUA scripts are hard to
change and debug. Once you run this file, you'll override the internal
versions with the ones specified here...

--]]

------------------------------------------------------------------------------
-- make_note_column_iter

local function make_note_column_iter(song, line_iter, visible_columns_only)
assert(type(line_iter) == 'function')
visible_columns_only = visible_columns_only or false

local pos, line = line_iter()

if (pos == nil or line == nil) then
return nil
end

local columns = line.note_columns
local column_count = #columns

if visible_columns_only then
column_count = song.tracks[
pos.track].visible_note_columns
end

-- we start by increasing the column
pos.column = 0

return function()
pos.column = pos.column + 1

if (pos.column <= column_count) then
return pos, columns[pos.column]
end

-- loop until we found a line with visible columns
while true do
pos, line = line_iter()

if (pos == nil or line == nil) then
return nil
end

columns = line.note_columns
column_count = #columns

if visible_columns_only then
column_count = song.tracks[
pos.track].visible_note_columns
end

if (column_count > 0) then
pos.column = 1
return pos, columns[pos.column]
end
end

return nil
end
end

-- make_effect_column_iter

local function make_effect_column_iter(song, line_iter, visible_columns_only)
assert(type(line_iter) == 'function')
visible_columns_only = visible_columns_only or false

local pos, line = line_iter()

if (pos == nil or line == nil) then
return nil
end

local columns = line.effect_columns
local column_count = #columns

if visible_columns_only then
column_count = song.tracks[
pos.track].visible_effect_columns
end

-- we start by increasing the column
pos.column = 0

return function()
pos.column = pos.column + 1

if (pos.column <= column_count) then
return pos, columns[pos.column]
end

-- loop until we found a line with visible columns
while true do
pos, line = line_iter()

if (pos == nil or line == nil) then
return nil
end

columns = line.effect_columns
column_count = #columns

if visible_columns_only then
column_count = song.tracks[
pos.track].visible_effect_columns
end

if (column_count > 0) then
pos.column = 1
return pos, columns[pos.column]
end
end

return nil
end
end

------------------------------------------------------------------------------
-- renoise.PatternIterator:lines_in_song
------------------------------------------------------------------------------

function renoise.PatternIterator:lines_in_song(visible_patterns_only)
visible_patterns_only = visible_patterns_only or true

local pattern_order = {}
if visible_patterns_only then
local pattern_sequence = self.song.sequencer.pattern_sequence
local referenced_patterns = {}

for seq_index, pattern_index in pairs(pattern_sequence) do
if not referenced_patterns[pattern_index] then
referenced_patterns[pattern_index] = true
pattern_order[#pattern_order + 1] = pattern_index
end
end
else
for pattern_index = 1,#self.song.patterns do
pattern_order[#pattern_order + 1] = pattern_index
end
end

local pattern_order_index = 1
local start_pos = { pattern = pattern_order[1], track = 1, line = 1 }
local pos = { pattern = pattern_order[1], track = 1, line = 1 }

local patterns = self.song.patterns
local pattern = patterns[pos.pattern]
local pattern_tracks = pattern.tracks
local pattern_track = pattern_tracks[pos.track]

-- we start by increasing the line
start_pos.line = start_pos.line - 1
pos.line = pos.line - 1

local function line_iter()
pos.line = pos.line + 1

if pos.line > pattern.number_of_lines then
pos.line = 1; pos.track = pos.track + 1

if pos.track > #pattern_tracks then
pos.track = 1; pattern_order_index = pattern_order_index + 1

if pattern_order_index > #pattern_order then
-- completed: reset and stop
pattern_order_index = 1
pos.pattern = start_pos.pattern
pos.track = start_pos.track
pos.line = start_pos.line

pattern = patterns[pos.pattern]
pattern_tracks = pattern.tracks
pattern_track = pattern_tracks[pos.track]
return nil

else
-- new pattern
pos.pattern = pattern_order[pattern_order_index]

pattern = patterns[pos.pattern]
pattern_tracks = pattern.tracks
pattern_track = pattern_tracks[pos.track]
end

else
-- new track
pattern_track = pattern_tracks[pos.track]
end

else
-- new line
end

return pos, pattern_track:line(pos.line)
end

return line_iter, self
end

-- note_columns_in_song

function renoise.PatternIterator:note_columns_in_song(visible_only)
return make_note_column_iter(self.song, self:lines_in_song(
visible_only), visible_only)
end

-- effect_columns_in_song

function renoise.PatternIterator:effect_columns_in_song(visible_only)
return make_effect_column_iter(self.song, self:lines_in_song(
visible_only), visible_only)
end

------------------------------------------------------------------------------
-- renoise.PatternIterator:lines_in_pattern
------------------------------------------------------------------------------

function renoise.PatternIterator:lines_in_pattern(pattern_index)
assert(type(pattern_index) == 'number', ('pattern_index parameter: ' ..
'expected an index (a number), got a \'%s\' object'):format(type(pattern_index)))

local start_pos = { pattern = pattern_index, track = 1, line = 1 }
local pos = { pattern = pattern_index, track = 1, line = 1 }

local pattern = self.song.patterns[pos.pattern]
local pattern_tracks = pattern.tracks
local pattern_track = pattern_tracks[pos.track]

-- we start by increasing the line
start_pos.line = start_pos.line - 1
pos.line = pos.line - 1

local function line_iter()
pos.line = pos.line + 1

if pos.line > pattern.number_of_lines then
pos.line = 1; pos.track = pos.track + 1

if pos.track > #pattern_tracks then
-- completed: reset and stop
pos.track = start_pos.track
pos.line = start_pos.line

pattern_track = pattern_tracks[pos.track]
return nil

else
-- new track
pattern_track = pattern_tracks[pos.track]
end

else
-- new line
end

return pos, pattern_track:line(pos.line)
end

return line_iter, self
end

-- note_columns_in_pattern

function renoise.PatternIterator:note_columns_in_pattern(pattern_index, visible_only)
return make_note_column_iter(self.song, self:lines_in_pattern(
pattern_index), visible_only)
end

-- effect_columns_in_pattern

function renoise.PatternIterator:effect_columns_in_pattern(pattern_index, visible_only)
return make_effect_column_iter(self.song, self:lines_in_pattern(
pattern_index), visible_only)
end

------------------------------------------------------------------------------
-- renoise.PatternIterator:lines_in_track
------------------------------------------------------------------------------

function renoise.PatternIterator:lines_in_track(track_index, visible_patterns_only)
assert(type(track_index) == 'number', ('track_index parameter: ' ..
'expected an index (a number), got a \'%s\' object'):format(type(track_index)))

visible_patterns_only = visible_patterns_only or true

local pattern_order = {}
if visible_patterns_only then
local pattern_sequence = self.song.sequencer.pattern_sequence
local referenced_patterns = {}

for seq_index, pattern_index in pairs(pattern_sequence) do
if not referenced_patterns[pattern_index] then
referenced_patterns[pattern_index] = true
pattern_order[#pattern_order + 1] = pattern_index
end
end
else
for pattern_index = 1,#self.song.patterns do
pattern_order[#pattern_order + 1] = pattern_index
end
end

local pattern_order_index = 1
local start_pos = { pattern = pattern_order[1], track = track_index, line = 1 }
local pos = { pattern = pattern_order[1], track = track_index, line = 1 }

local patterns = self.song.patterns
local pattern = patterns[pos.pattern]
local pattern_tracks = pattern.tracks
local pattern_track = pattern_tracks[pos.track]

-- we start by increasing the line
start_pos.line = start_pos.line - 1
pos.line = pos.line - 1

local function line_iter()
pos.line = pos.line + 1

if pos.line > pattern.number_of_lines then
pos.line = 1; pattern_order_index = pattern_order_index + 1

if pattern_order_index > #pattern_order then
-- completed: reset and stop
pattern_order_index = 1
pos.pattern = start_pos.pattern
pos.line = start_pos.line

pattern = patterns[pos.pattern]
pattern_tracks = pattern.tracks
pattern_track = pattern_tracks[pos.track]
return nil

else
-- new pattern
pos.pattern = pattern_order[pattern_order_index]

pattern = patterns[pos.pattern]
pattern_tracks = pattern.tracks
pattern_track = pattern_tracks[pos.track]
end

else
-- new line
end

return pos, pattern_track:line(pos.line)
end

return line_iter, self
end

-- note_columns_in_track

function renoise.PatternIterator:note_columns_in_track(track_index, visible_only)
return make_note_column_iter(self.song, self:lines_in_track(
track_index, visible_only), visible_only)
end

-- effect_columns_in_track

function renoise.PatternIterator:effect_columns_in_track(track_index, visible_only)
return make_effect_column_iter(self.song, self:lines_in_track(
track_index, visible_only), visible_only)
end

------------------------------------------------------------------------------
-- renoise.PatternIterator:lines_in_pattern_track
------------------------------------------------------------------------------

function renoise.PatternIterator:lines_in_pattern_track(pattern_index, track_index)
assert(type(pattern_index) == 'number', ('pattern_index parameter: ' ..
'expected an index (a number), got a \'%s\' object'):format(type(pattern_index)))
assert(type(track_index) == 'number', ('track_index parameter: ' ..
'expected an index (a number), got a \%s\ object'):format(type(track_index)))

local start_pos = { pattern = pattern_index, track = track_index, line = 1 }
local pos = { pattern = pattern_index, track = track_index, line = 1 }

local pattern = self.song.patterns[pos.pattern]
local pattern_tracks = pattern.tracks
local pattern_track = pattern_tracks[pos.track]

-- we start by increasing the line
start_pos.line = start_pos.line - 1
pos.line = pos.line - 1

local function line_iter()
pos.line = pos.line + 1

if pos.line > pattern.number_of_lines then
-- completed: reset and stop
pos.line = start_pos.line
return nil

else
-- new line
end

return pos, pattern_track:line(pos.line)
end

return line_iter, self
end

-- note_columns_in_pattern_track

function renoise.PatternIterator:note_columns_in_pattern_track(
pattern_index, track_index, visible_only)
return make_note_column_iter(self.song, self:lines_in_pattern_track(
pattern_index, track_index), visible_only)
end

-- effect_columns_in_pattern_track

function renoise.PatternIterator:effect_columns_in_pattern_track(
pattern_index, track_index, visible_only)
return make_effect_column_iter(self.song, self:lines_in_pattern_track(
pattern_index, track_index), visible_only)
end

--[[--------------------------------------------------------------------------
--------------------------------------------------------------------------]]--
13

thanks!