New Tool (3.0,3.1): xStream

I feel weird posting a bunch of times in a row, but I prefer to keep the individual posts to one thought rather than having them all jumbled up. Anyway…

I have questions about userdata (which seems to mean two different things in xStream).

  1. What are the userdata aspects used for?

From what I can tell, it’s mostly a way of defining a constant, so you don’t have to have a variable declaration in the main code?

I have found it useful for functions – I can extract a few lines out to a function and call that. But for the numbers and strings I don’t really get it, unless it really is just a convenient way to define a value out of sight.

  1. What’s the Userdata folder option for?

I assumed that once I set it, saving my model would save to that folder. But I’ve set the Userdata folder option, and when I save my new model it saves to the tool’s userdata folder along with the pre-built models.

  1. Regarding the first bug, my wild assumption is that danoise is doing some appending to the sandbox code with a semi-colon instead of a newline. This would mean that the appending, probably a “write line” function and some other stuff, would get commented out if the user code is ending with a comment. Perhaps you can end on a comment if you at least add a linebreak after?

  2. “if not vel then” should be invalid if I’m not mistaken. You can’t test a variable that doesn’t exist. “if xinc == 0 then” is indeed the way to go.

Leaving the rest to the expert!

  1. No doesn’t work with a linebreak after, but you can add any valid lua code. So I just added a “local dontbreak” as the last line, and then I can comment out whatever.

  2. I’m pretty sure “if not vel then” is the way to test if it’s undefined. It was used in a suggestion a few pages back, and seems to do an initial setting of the variable. But it’s checking based on definition.


Here’s a fun little gotcha… let’s say you want the model to clear the line before doing anything. Don’t do:

xline = EMPTY_XLINE
-- your stuff here

Because any changes you make to xline will fill up the EMPTY_XLINE and you won’t have any more empty xlines anymore :slight_smile: You’ll have to reload the tool.

Instead you should do:

xline = table.rcopy(EMPTY_XLINE)

And that will clear out the xline without screwing with the EMPTY_XLINE constant.

Yeah! A typical LUA issue - when you lose track if a variable is a reference or an actual value. I gues every assignment pointing to an object or a table will just be a reference. A dumb rule of thumb for remembering how it works might be the following: If you can display the entire content of something with the simple print() function, the value will be copied and not referenced when used in an assignment.

  1. Ah, ok. I guess that the xstream sandbox environment is using the __newindex metamethod to return nil if a variable doesn’t exist. Makes sense to simplify syntax within a sandbox (normally you would prepare by just doing “local vel” in a higher scope).

That’s a good rule of thumb, thanks.

So, this tool makes me want to never sleep again.

This model doesn’t work:

-- Use this as a template for your own creations.
column = xline.note_columns[1]
column.note_string = "C-4"
-- break!

Maybe I’m dense but that seems like a bug.

Definitely. Thanks for spotting it. Same with the custom user-folder issue you reported (will investigate).

  1. What are the userdata aspects used for?

From what I can tell, it’s mostly a way of defining a constant, so you don’t have to have a variable declaration in the main code?
I have found it useful for functions – I can extract a few lines out to a function and call that. But for the numbers and strings I don’t really get it, unless it really is just a convenient way to define a value out of sight.

Yes, unlike variables declared in the main function, userdata is only “set once” and therefore, a better candidate for variables that change over time (as in your original question/observation).

Apart from that, it’s really just a handy way to specify values and functions outside of the main function, cleaning things up a bit.

For example, the Arpeggiator model would be a complete mess without userdata.

I guess that the xstream sandbox environment is using the __newindex metamethod to return nil if a variable doesn’t exist

Yep, you got it. The sandbox is trying to simplify things a bit.

Here’s a fun little gotcha…

Hmm - this could be changed so there would be no need for table.rcopy - xStream shouldsimply always deliver a deep copy.

At least, I can’t think of a scenario where it makes sense to modify/override EMPTY_XLINE and other constants.

Yes, unlike variables declared in the main function, userdata is only “set once” and therefore, a better candidate for variables that change over time (as in your original question/observation).

Apart from that, it’s really just a handy way to specify values and functions outside of the main function, cleaning things up a bit.

For example, the Arpeggiator model would be a complete mess without userdata.

Interesting… I’m not following the userdata is only “set once” part. Well maybe I am. Does that mean that you don’t have to do:

if not myvar then
  myvar = 2
end

kinda thing? You just set the value in the userdata and it essentially does that?

Also, how come things went all wacky when I modified the args value directly?

Hmm - this could be changed so there would be no need for table.rcopy - xStream shouldsimply always deliver a deep copy.

At least, I can’t think of a scenario where it makes sense to modify/override EMPTY_XLINE and other constants.

I agree. It was definitely a PITA when I did it. I suspect that as more people use xStream and discover new patterns, those will make their way into the library. For this case, I think something like xline:empty() would do the job. I don’t know much lua so I don’t know if that’s how you’d want to go about it, but it makes sense to me.

You’re probably going to get annoyed with me as I use it and come up with feature requests (maybe I can figure this stuff out well enough to send pull requests). Anyway, a couple things…

  • I’d love for the number-displayed-as-note to have left and right arrow buttons to decrease and increase the note value by 1. Dragging the slider isn’t terrible but for me it’s faster and more precise to use arrow buttons for small quick changes.
  • I’d be interested to know if there’s some way to do a sort of data-binding for arg values, particularly with the chooser. For example, Demo-Notes-Scale defines its keys and scales, duplicating most of the information from xScale, and requiring that the keys be positioned the same as xScale. I’d like to somehow say that the possible values come from xScale. Maybe that’s already possible with the bind/poll setting?
  • It’d be nice if the terminal editor didn’t show an error every time I typed… I like that it’s checking for syntax, not sure how to make it less noisy though while I’m typing.

This is really cool, you’re pushing Renoise forward in amazing ways! xLib in particular looks like it’s going to be incredibly useful.


edit: It looks like we don’t have access to xNoteColumn? It’s got a couple functionsnote_string_to_value and note_value_to_string that would come in handy… is there a way to access this from xStream, perhaps I’m just doing it wrong? It says xNoteColumn is nil.

This is really neat. I’m working on a little chord generating tool. It’s pretty nice for messing with inversions. It wasn’t all that difficult, either…

Two things I want to add are automatic voice-leading, and some kind of text-based progression generator so you could type in “Cmaj7 G7 Am7 F7” kinda thing… those are for another day though :slight_smile:

Here’s what it looks like from the outside:

6995 Screen Shot 2016-09-15 at 9.31.57 PM.png

xline = table.rcopy(EMPTY_XLINE)

local notes = {
  data.noteRoot(),
  data.noteThird(),
  data.noteFifth(),
  data.noteSeventh(),
  data.noteBass(),
}

table.sort(notes)

for i=1,#notes do
  xline.note_columns[i].note_value = notes[i]
end

print("success!")

Cool, pat! I started making a chord class / manipulator that was intended to be used with xStream. It’s not finished, but if you want to use anything or draw inspiration from it you can have a look at https://github.com/snabeljoel/Chord-manipulation-model . It was a different take than yours, primarily meant to eventually use a second track as a raw source track and then generate various figures with a guiding chord track. (Kind of possible when hacking xStream a bit).

(However, i discontinued the project for xStream and I’m aiming to implement it in another tool I’m working on where it might fit the framework and workflow better. xStream is awesome for generating a stream of pattern data in a single track, though)

PS. Good luck with the voice leading :wink:

I’d love for the number-displayed-as-note to have left and right arrow buttons to decrease and increase the note value by 1. Dragging the slider isn’t terrible but for me it’s faster and more precise to use arrow buttons for small quick changes.

I think it makes sense too. Will change this for the next version :smiley:

I’d be interested to know if there’s some way to do a sort of data-binding for arg values, particularly with the chooser. For example, Demo-Notes-Scale defines its keys and scales, duplicating most of the information from xScale, and requiring that the keys be positioned the same as xScale. I’d like to somehow say that the possible values come from xScale.

Um, yeah. That model was created a while back, just as userdata was introduced and before xScale was a reality.
There surely is better ways to initialize things, but… it works :smiley:

It’d be nice if the terminal editor didn’t show an error every time I typed… I like that it’s checking for syntax, not sure how to make it less noisy though while I’m typing.

Again, makes sense. Could be an option, “verbose syntax check” or something…

It looks like we don’t have access to xNoteColumn? It’s got a couple functionsnote_string_to_value and note_value_to_string that would come in handy… is there a way to access this from xStream

Indeed, no we don’t have access but I’ll add it to the next update.

For now, you can do this yourself - if you open the file xStreamModel.lua and look for ‘props_table’ then you’ll see a list of static classes that are exposed to models.
Just copy/paste/modify one of the existing entries to add the xNoteColumn class.

Does that mean that you don’t have to do:

if not myvar then
myvar = 2
end

kinda thing? You just set the value in the userdata and it essentially does that?

Also, how come things went all wacky when I modified the args value directly?

Yes, you got it (reg. userdata). Not sure what you mean about wacky args though?

By the way, here is a model that Palo van Dalo shared with me.
It controls a 16-step note sequence with random notes in the selected scale.
Each step with a probability of being output, or not:

--[[===========================================================================
RandomScale.lua
===========================================================================]]--

return {
arguments = {
  {
      ["locked"] = false,
      ["name"] = "base_note",
      ["linked"] = false,
      ["value"] = 44.093589041096,
      ["properties"] = {
          ["min"] = 0,
          ["impacts_buffer"] = false,
          ["display_as"] = "note",
          ["max"] = 108,
      },
      ["description"] = "the base not to play",
  },
  {
      ["locked"] = false,
      ["name"] = "instrument",
      ["linked"] = false,
      ["value"] = 0,
      ["properties"] = {
          ["zero_based"] = false,
          ["max"] = 100,
          ["display_as"] = "integer",
          ["min"] = 0,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "base_mood",
      ["linked"] = false,
      ["value"] = 3,
      ["properties"] = {
          ["max"] = 4,
          ["zero_based"] = false,
          ["display_as"] = "integer",
          ["min"] = 1,
      },
      ["description"] = "- 1 major\n- 2 major 2\n- 3 major 3\n- 4 minor",
  },
  {
      ["locked"] = false,
      ["name"] = "note_1_prop",
      ["linked"] = false,
      ["value"] = 10,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_1_1_prop",
      ["linked"] = false,
      ["value"] = 0,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_1_2_prop",
      ["linked"] = false,
      ["value"] = 0,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_1_3_prop",
      ["linked"] = false,
      ["value"] = 3.1506849315068,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_2_prop",
      ["linked"] = false,
      ["value"] = 0,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_2_1_prop",
      ["linked"] = false,
      ["value"] = 7.6712328767123,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_2_2_prop",
      ["linked"] = false,
      ["value"] = 5.4794520547945,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_2_3_prop",
      ["linked"] = false,
      ["value"] = 1.7808219178082,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_3_prop",
      ["linked"] = false,
      ["value"] = 10,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_3_1_prop",
      ["linked"] = false,
      ["value"] = 0,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_3_2_prop",
      ["linked"] = false,
      ["value"] = 1.0958904109589,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_3_3_prop",
      ["linked"] = false,
      ["value"] = 0,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_4_prop",
      ["linked"] = false,
      ["value"] = 8.0821917808219,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_4_1_prop",
      ["linked"] = false,
      ["value"] = 0,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_4_2_prop",
      ["linked"] = false,
      ["value"] = 2.6027397260274,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
  {
      ["locked"] = false,
      ["name"] = "note_4_3_prop",
      ["linked"] = false,
      ["value"] = 0,
      ["properties"] = {
          ["min"] = 0,
          ["display_as"] = "minislider",
          ["max"] = 10,
      },
      ["description"] = "",
  },
},
presets = {
},
data = {
},
events = {
},
options = {
 color = 0x000000,
},
callback = [[
-- ------
-- propability 1 to 10 if true
-- ------

function random( percent )
  return ( math.random(0, 9) < percent )
end

-- ------
-- set note or off or empty
-- ------

-- note 

function note_prop( step_size, line, prop )
  if ( xpos.line % step_size == line )
  then
    return random(prop) 
  else
    return false
  end
end

function note_or_not()
  return ( 
    note_prop( 16, 1, args.note_1_prop ) or
    note_prop( 16, 2, args.note_1_1_prop ) or
    note_prop( 16, 3, args.note_1_2_prop ) or
    note_prop( 16, 4, args.note_1_3_prop ) or
    note_prop( 16, 5, args.note_2_prop ) or
    note_prop( 16, 6, args.note_2_1_prop ) or
    note_prop( 16, 7, args.note_2_2_prop ) or
    note_prop( 16, 8, args.note_2_3_prop ) or
    note_prop( 16, 9, args.note_3_prop ) or
    note_prop( 16, 11, args.note_3_1_prop) or
    note_prop( 16, 10, args.note_3_2_prop) or
    note_prop( 16, 12, args.note_3_3_prop) or
    note_prop( 16, 13, args.note_4_prop) or
    note_prop( 16, 14, args.note_4_1_prop) or
    note_prop( 16, 15, args.note_4_2_prop) or
    note_prop( 16, 0, args.note_4_3_prop) )
end

-- off

function off_or_not()
  return false
end    

-- ------
-- choose note
-- ------

local major = { 0, 2, 4, 5, 7, 9, 11 }
-- base chord is more likely
local major2 = { 0, 0, 2, 4, 4, 5, 7, 7, 9, 11 } 
-- base chord is even more likely
local major3 = { 0, 0, 0, 2, 4, 4, 4, 5, 7, 7, 7, 9, 11 } 

local minor = { 0, 2, 3, 5, 7, 8, 10 }
local base_note_value = args.base_note

function random_minor_value()
  return base_note_value + minor[math.random(1,7)]
end

function random_major2_value()
  return base_note_value + major2[math.random(1,10)]
end

function random_major3_value()
  return base_note_value + major3[math.random(1,13)]
end

function random_major_value()
  return base_note_value + major[math.random(1,7)]
end

function note_value()
  if (args.base_mood == 1)
  then 
    return random_major_value()
  elseif (args.base_mood == 2)
  then 
    return random_major2_value()
  elseif (args.base_mood == 3)
  then 
    return random_major3_value()
  else
    return random_minor_value()
  end
end

-- -----
-- main
-- -----

if( note_or_not() ) then

  xline.note_columns[1] = {
    note_value = note_value(),
    instrument_value = args.instrument
  }

elseif( off_or_not() ) then
  
  xline.note_columns[1] = {
    note_value = 120, 
    instrument_value = args.instrument
  }  
  
else

  xline.note_columns[1] = {
    note_value = 121,
    instrument_value = 255
  }  

end
]],
}

Before including with xStream I probably want to tweak it a bit (most importantly: expand the number of possible scales - having only four hard-coded scales seems a bit limited).

Cool, pat! I started making a chord class / manipulator that was intended to be used with xStream. It’s not finished, but if you want to use anything or draw inspiration from it you can have a look at https://github.com/snabeljoel/Chord-manipulation-model . It was a different take than yours, primarily meant to eventually use a second track as a raw source track and then generate various figures with a guiding chord track. (Kind of possible when hacking xStream a bit).

Nice, thanks for that. Recognizing chords is on my list so that’ll come in handy, as well as open voicings. Good stuff in there!

Here’s my renoise models repo

Yes, you got it (reg. userdata). Not sure what you mean about wacky args though?

I’m referring to this:

I’m struggling to understand how args work. I’m trying to change a value over time. I’m outputting a note value along with a volume, and decrementing the volume with each iteration. I’m using offline mode, “apply to selected track.” Here’s my code:

column = xline.note_columns[1]
column.note_string = "C-4"
column.instrument_string = "00"
column.volume_value = args.velocity
if args.velocity > 10 then
args.velocity = args.velocity - 1
end

Starting with a velocity value of 7F (hex), I would expect it to write one line with a velocity of 7F, the next with 7E, and so on until it flatlines at 0A. Instead, it’s writing only one line – the very last line of the pattern – with a velocity of 40. This is a 40 line pattern.

I’m referring to this:

OK - I guess you’re writing to the track in one go (offline mode)? Because, it works when streaming.

I’ll look into why it behaves differently in offline mode, but generally speaking, it’s more appropriate to use xinc or xpos.line for something like this.

Hi folks. I was desperately looking for a DAW with integrated scripting and found Renoise and Xstream. Both are very exciting and I’m demoing them now. Ideally, I was hoping to create patterns that evolve through MIDI controller data using one model per track. This approach is similar to tidalcycles, an amazing live coding environment in Haskell. Since Xstream sends output to the current track, A user on IRC recommended cloning the tool (install multiple copies) in order to have a model per track. I’m going to try that now. However, perhaps multi-track functionality could be added to the list of feature requests? Or perhaps someone has another suggestion of how to attach small generative and interactive scripts to each track?

Thank you,

-d.vyd

+10 for going into the multi-track realm!

Another request:

Button for track-in-song wide rendering (like the TRK button). I’ve actually used xstream more for rendering/generating data rather than streaming it.

Joule on IRC sent me this amazing code that can send to arbitrary tracks:

local yline = renoise.song():pattern(renoise.song().sequencer.pattern_sequence[xpos.sequence]):track(1):line(xpos.line)

yline.note_columns[1].note_value = 55

I’m trying to add MIDI control to the model. I’ve created an argument in xStream and mapped that argument in the Song–> MIDI mapping dialog. However, I cannot seem to get the argument to respond to MIDI even though the MIDI learn seems to have worked. Do I need to poll or bind the argument? If not, what am I doing wrong here?

local yline = renoise.song():pattern(renoise.song().sequencer.pattern_sequence[xpos.sequence]):track(3):line(xpos.line)

yline.note_columns[1].note_value = args.arg1

-d.vyd

PS: 03/31/2017–I’m still fiddling with this but have not made any progress. I noticed there are MIDI settings within the xStream window–but turning on MIDI input there also did not solve the problem.

I was able to control the “interval” argument of the Demo-Periodic Output model with MIDI, so the problem seems to be in my own script. I’ll study the Periodic Output script and try again.

Wondering if you made any progress?

I took a quick look, but without access to the actual script (model) that you’re working on, it’s a bit difficult for me to offer any assistance.

I noticed there are MIDI settings within the xStream window–but turning on MIDI input there also did not solve the problem.

This is perhaps not obvious, but those settings are internal to xStream.

In other words, if you see a control which “lights up” as you bring up the Renoise MIDI mapping dialog, this is completely separate from the internal xStream MIDI/OSC handling.

The internal MIDI can be used for more advanced control, when it’s not sufficient to MIDI map in the normal way. For example, song-position or sysex messages can be handled from there…

Hi Danoise, thank you for checking in. I did get MIDI to work with your examples. But, I think the learning curve for Renoise, the tracker paradigm in general, and xStream together is too much for me right now. Instead, I’ve made progress creating a sequencer that works with my hardware controller in Pure Data. I’m sending the output to my DAW (MuLab) to generate audio.

From a workflow standpoint I imagine this tool working in a phrase mode.

Imaginary usage example1: The Tool is “bind” to current instrument. I get a one phase as input and output is another phrase. Settings like phrase in/out and maybe what “program” is run is stored inside instrument. Changing instrument deactivates the “program” execution and reads those settings.

I`m writing this as a encouragement and try to do the actual work. Hopefully this makes sense and be continued.