New Tool (3.0,3.1): xStream

I can see how a lean & mean syntax for a “pulse generator” might be appealing, but it’s (IMHO) more important to keep that part as flexible and open as possible. For example, if we take a cue from our talks about the Euclidean rhythms model, this is a promise of the (planned) stacked models feature - the almighty Euclidean pulse generator.

Just some input… I would probably “abuse” stacked models for generating multiple tracks rather than stacking up different criteria for a single track. I think I’d prefer one model per generated track, but that the coding of models instead was simplified as much as possible.

About an oscillator class… 1) A simple oscillator class would be very similar with how we use the LFO device. It’s a proven concept. 2) An oscillator class could be used not only for generating “note hits”, but also to set values in any of the other columns.

osc_note = oscillator("pulse", 16, 0, 0, 1) -- type (pulse is not the same as square), lines, ticks, start value, end value .. construct only if object doesn't exist, else just return value
osc_vol = oscillator("saw", 64, 0, 255, 0)
-- osc_comb = c_oscillator("multiply", first_osc, second_osc) .. something like this for combining oscillators?

if osc_note then
  xline.note_column[1] = blabla
  xline.note_column[1].volume_value = osc_vol
  osc_vol:reset() -- more functions possible? :reverse() could reverse the oscillator at its current position .. :reset([percentage_of_full_cycle]) .. :offset(jump_percentage_of_full_cycle)
end

Everything here would be pretty easy to make. Basic maths on xinc, except for the :reverse() function that would be a bit trickier/uglier.

Just some input… I would probably “abuse” stacked models for generating multiple tracks rather than stacking up different criterias for a single track. I think I’d prefer one model per generated track, but that the coding of models instead was simplified as much as possible.

Yes, abuse would be the right term. Surely we can come up with better solutions?

Stacked models = ability to run multiple models, as if (from a user perspective) they were a single unit. So not much additional UI complexity, just more “modularity”.

One very obvious purpose would be to use a simple model as note generator, but chain reactions would be possible too. Very interesting stuff!!

Technically, the models would share a single buffer, and the xline would have an accompanying stack property so you could access the xline from any level in the stack

Multiple instances = the ability to run multiple models side by side, each one targeting a specific track (and prominently featured in the UI).

This is basically xStream XL - a multitrack version.Technically, each instance would have it’s own buffer and shared timing, but not much else.

I’ve been looking into simple ways of controlling/automating those xstream instances too - primarily via pattern commands.

An oscillator class would be very similar with how we use the LFO device. It’s a proven concept

Exactly - the standalone Renoise LFO device would suck if you couldn’t hook it up to key/velocity-trackers, or automate it. So, it’s more about much integration it can enjoy - as in, what the tool can offer in terms of event handlers, automation, notifications and such.

Not sure I can contribute much since I’m still trying to wrap my head around the practicallities of stacked models :slight_smile: I am guessing that these would not just be generating pattern data in a serial fashion, from one model to the other until it reaches the last model generating the finite pattern data? Rather, I guess that these models would be just parallell ‘sandboxes’ sharing the same variables? That would imo imply that you would need to 1) do a little bit of formalizing as of how to structure models depending on their type (note value generator, note hit/rhythm generator et c), and 2) end all chains with a “write pattern data” model.

Am I complicating matters unnecessarily?

Well, in my mind it really is that simple - passing data (xlines) from one model to the next until we reach the last one, at which point the pattern data is written.

And ideally, one model would not need to know about where that data came from. Args would still be args - the syntax would not have to change. Instead we add a stack property that you can query to obtain the xline from any model - right down to the original one.Want to access an argument from another model - again, the stack delivers_._

Here are a couple of ideas for how a stack could be used:

  • Post-processing models which offer basic functionality such as having a master volume for notes, or remapping notes to a specific instrument etc. The generic kind of stuff that currently just bloats up the size of our models.
  • Currently, the Exponential Delay model responds to already-existing notes in the first note column. It leaves that column unchanged, and creates its output in columns 2-12.With a stacked model approach, you could use the Euclidean model as a generator for the first note column, and the Exponential Delay would respond to that, not being aware that the note had just been “handed over” to it.
  • Supporting “mute” notes for the Euclidean model - e.g. to blank out the kick when there was a snare hit, etc. Basically, what you mentioned that you would like to implement. This sort of stuff should be possible with stacking too, by adding a simple "post-processing"model, specifically tailored for that purpose.
  • With direct buffer access, scheduling xlines as a response to input from another model could create interesting chain reactions.

That sounds good indeed.

I am thinking that a ‘good practice’ way of making models under such circumstances would be to provide an args.model_input_line which would be any of { xline, model_1_output, model_2_output, et c }. Then you would have a flexible system (capable of “paralell processing”) as well as being common sense GUI wise. It would not matter in what order you insert models, as long as you select the correct input. This would, however, call for finalizing the chain with a “write pattern data” model (or hell… using a midi output model, making xstream a sequencer on its own)

I am guessing your stack idea would be doing something like this, but just thought it would be good to reveal this input selection clearly as a select box in the GUI (with args?).

You might want one chain to generate the volume column and one chain to generate note data. These would then be finalized with a “merge and output” model providing two args.input, as well as some args ticks that would for example handle tasks like “only output volume data when there is note data” (only input 2 on input 1), and perhaps even revealing some boolean operations for merging.

In addition, the possibility of grouping chains would double as multi-track capability, starting each chain with an “input model” that basically does thischain_xline = rns:pattern(xpos.pattern_index):track( args.track_index ):line(xpos.line) – (too hackish? breaking some of the purpose of xstream/xline?)

I imagine every model would need to be finalized with a statement like “output = my_line” in that case. (Or rather, the name of the internal xline variable is being fixed)

While MIDI output for xlines might sound tempting, it wouldn’t really work as you are asking a tool to output MIDI on it’s own initiative (== sluggish timing).
This was actually the raison d’etre for this tool, to overcome this limitation while not having to re-invent the wheel over again - having done multiple "streaming"sequencers in the past.

You might want one chain to generate the volume column and one chain to generate note data. These would then be finalized with a “merge and output” model providing two args.input, as well as some args ticks that would for example handle tasks like “only output volume data when there is note data” (only input 2 on input 1), and perhaps even revealing some boolean operations for merging.

Perhaps dancing would a fitting analogy here. As long as you don’t step on your partners toes, things are good. Same with stacked models: if you could not access the stack, you would have to make sure that model B does not “blindly” overwrite the output of model A.

But with access to the stack, even if a model does “blindly” overwrite the output, a third model C could in fact separate and clean up the output from the first and second one.
So, in this case model A and model B would be completely normal models without any knowledge of a stack or anything, while the third one would somehow know about both.

At least, that’s how I imagine it would work. Focus is on maximum flexibility, basically.

I imagine every model would need to be finalized with a statement like “output = my_line” in that case

Unless you’d want to hand over output to a particular model at a later point in the stack, IMOthis should all be done automatically.

But in some special cases, having control over exactly where the output is handed over is definitely valid.

I am not quite understanding the stack idea, so maybe you’ve already thought it thru in a better way.

Anyway… I am guessing you want to structure these stacks in a way that is fairly compatible with how xstream models work right now. If you want to make a simple model current-style, writing to xline, it should still be possible with the same code. The user should not have to configure anything extra in that case, and all presets should work in both single or stacked mode (?).

I am guessing the most obvious way would be to provide a select box on each model, stating exactly what xline is to THAT model - 1) the real xline (default), 2) the xline of track x/y, 3) a copy of any of these, 3) the xline from model 1, 2 or 3.

Well, the idea is still forming in my head and so, as time goes,I can only become better at explaining it.

I am guessing the most obvious way would be to provide a select box on each model, stating exactly what xline is to THAT model - 1) the real xline (default), 2) the xline of track x/y, 3) a copy of any of these, 3) the xline from model 1, 2 or 3.

Not really. From an model-authoring POV, it should be a simple concept to work with. The xline you are reading at any time is native to that model - it simply contains whatever gets passed. First model in the stack gets the pattern content (like how it currently works), second one gets the output from the first model in the stack, third one the output from the second and so forth. So in most cases, the model simply works “as-is”, and there’s nothing new to learn. It’s only when you actually need/want direct accessto the stack that this becomes something to deal with. Which is how it’s supposed to work.

The way I have envisioned it, a stacked model should appear alongside normal models - meaning, you can select it from the usual model popup menu. This makes sense, since (from a user perspective) a stacked model doesn’t actually bring anything new to the table. Everything that stacked models can offer , you can already do in xStream now, it’s just that complex model tend to get unwieldy and the tool is not meant as a full-blown IDE. You might even argue that, as arguments can become spread across a number of models, it’s_less_convenient to a casual user than a non-stacked model … because you might have to select a model before you can access the visual controls.

Not exactly sure how to deal with import and export of stacks though - we are basically getting into the territory of dependencies (shudder), as a stack would be referencing a number of models. But I can’t see an easy way to avoid this. So exporting a stack would probably mean exporting a bundle of model, which - when imported, will “install” these models before the stack can be instantiated. Or something like that.

So, it’s the actual implementation details that are hairy and will take some time to get right. Perhaps you can see why it’s a feature that I have (so far) put on the backburner. As a programmer, it brings some cool opportunities for making models more modular. But it also brings some complexity to the table - unfortunately, on the user part. I’m hoping this can somehow be solved without too much hassle.

That sounds great. I assume the user will be able to redefine xline to anything at the top of the models user code?

I imagine that a stack of models would contain everything, i e not referencing any kind of “model library”. Once a model is added, it would be copied into the stack as something unique in that stack.

The only drawback I can see is that the ‘core’ of a model might get updated. The user would then have to copy/paste and update all stacks (if it’s really that relevant, which it probably seldom is). No biggie imo.

Yes, the stack/chain function is mainly a GUI thing, but an important one I think.

With this you could for example do:

– setting up
if xinc == 1 then volume_osc = osc.create(sine, 16, 0, 255) end
– using
xline.note_columns[1].volume_value = volume_osc:value()

Update: I think it’s almost done here. Might need some bug fixing and tidying things up. + Documentation (it’s quite flexible)

Click to view contents
local xinc -- for testing only
local osc = {
  func = {
    sine = {
      user_abr = { "sine", "sin" },
      calc = function(phase) return (math.sin(math.rad(360 * phase)) / 2 + 0.5) end
    },
    square = {
      user_abr = { "square", "pulse", "sqr", "pul" },
      calc = function(phase) if (phase < 0.5) then return 1 else return 0 end end
    },
    random = {
      user_abr = { "random", "rnd", "noise" },
      calc = function(phase) return math.random() end
    },
    triangle = {
      user_abr = { "tri", "triangle" },
      calc = function(phase) return math.abs(((phase+0.75) % 1) - 0.5) * 2 end
    },
    saw = {
      user_abr = { "saw", "ramp", "sawtooth" },
      calc = function(phase) return phase end
    },
    min = {
      user_abr = { "min" },
      calc = function() return 0 end
    },
    max = {
      user_abr = { "max" },
      calc = function() return 1 end
    }
    -- make a step sequencer here? or at least a toggler? maybe implement euclidean pulses?
  },
}
osc.get_index = function(user_type)
  for i_osc, osc in pairs(osc.func) do
    for i_abr, abr in ipairs(osc.user_abr) do
      if (abr == user_type) then return i_osc end
    end
  end
  return "min"
end
function osc:validate()
  return (self.type and self.freq and tonumber(self.freq) and self.min_value and tonumber(self.min_value)
          and self.max_value and tonumber(self.max_value))
end
function osc:runtime()
  return xinc - self.start_xinc
end
function osc:reset(phase)
  self.start_xinc, self.phase = xinc, phase or self.phase
end
function osc:value(scale_min, scale_max, resolution)
  if self then
    local phase = self:runtime() % self.freq / self.freq
    local min, max, res = scale_min or self.min_value, scale_max or self.max_value, resolution or self.resolution
    local value = osc.func[osc.get_index(self.type)].calc(phase) * (max-min) + min -- bad practice referencing osc template instead of using metatables? (lack of selfness?)
    return math.max(min, math.min(max, math.floor(value/res+0.500001) * res)) -- not sure why. bug in the universe/lua?
  end
end
function osc.create(type, freq, min_value, max_value, phase, resolution)
  local object
  if (type and freq) then
    object = {
      type = osc.get_index(type),
      freq = freq,
      min_value = min_value or 0,
      max_value = max_value or 255,
      resolution = resolution or 1,
      phase = phase or 0,
      start_xinc = xinc or 1
    }
    for k, val in pairs(osc) do
      if not (k == "func") or (k == "get_index") then -- again, instead of using metatables
        object[k] = val
      end
    end
  end
  if object:validate() then return object else return nil end
end
-- testing below
for i = 1, 64 do xinc = i
if xinc == 1 then my_osc = osc.create("tri", 32) end
print(my_osc:value())
end

EDIT:

Not sure danoise will find it useful enough, but I for sure will use it anyway :slight_smile:

Another (totally different) idea I got is a very small and simple helper function that could be available in xStream that I would find quite useful and that would possibly simplify some aspects of xStream coding even more. The following:

if on_update(xline.note_columns[1].note_value) then
xline.note_columns[1].effect_value = 10
end

Super simple syntax imo.

on_update() will return the value only when changed, otherwise it will return nil. The idea is also that the on_update() function would not only accept variable data, but even a whole object (excl methods).

It should be pretty easy to code. I imagine that the first time on_update is being ran, it will register the reference in an internal table (variables would be normal values and object/tables would be hash values). Registering/checking objects and tables would require some recursive scanning, sure. On each subsequent update it will compare the current value to what is cached. Not sure how to get the “uniqueness” of the parameter passed into the function, but it should be possible? Or maybe there is a better way already available for similar tasks?

EDIT 2: An alternative would be to hook up a document to xline properties, and make use of document observables? Observing xline is quite enough, I think, maybe allowing for something like this if possible:

if xline.note_columns[1].note_value:is_updated() == true . Or is something similar already possible?

IDEA 3:

Another simplifying idea, or at least one that adds flexibility, would be to provide not only xline but also something like xline(offset).

xline(16).note_columns[1].effect_value = 10

This is sometimes more convenient than 1) caching values for later use or 2) calculating the correct value with a possibly more intricate function (offsetting), 3) dealing with xpos.

A very simple practical example would be to generate a tracked delay in a second column. In this case xline(steps) would imo make more sense, or at least be easier, than storing data in a table for % search or fetching data from previous lines.

Just an idea for streamlining/simplifying model coding and improving readability, making new users feel even more welcome :slight_smile:

Handy LFO class for use with xStream (or adaptable to anything) can be found here:

https://github.com/snabeljoel/xStream-LFO-class

The usage in xStream would typically be something like below. Some other parameters are available as well. I’ll try posting a video later (mainly for hyping xStream!).

if xinc == 0 then
my_osc = LFO("sine", 16)
end

-- current_value = my_osc.value
scaled_value_for_parameters = my_osc(0, 255, 1) -- min, max, resolution

I will probably start working on something more elaborate that is more adapted to the parameters of the Renoise LFO/meta devices, and various pattern effects. By this xStream could be abused to make LFO/meta devices control pattern data in a more familiar fashion. For instance controlling pitch (with pitch effects) via an LFO meta device. This idea would work with realtime streaming only, though.

Handy LFO class for use with xStream (or adaptable to anything) can be found here:

Looking good! Should definitely be included in xStream once we have some test cases/demo :slight_smile:

“LFO” is also a better name for a class. You previously called it “osc”, I thought of pointing out that it’s more or less a convention to have class names in CamelCase.
Mostly because then, it’s easy to distinguish between variables and classes, and calls to static methods of the class itself.
Details, but worth pointing out.

I will probably start working on something more elaborate that is more adapted to the parameters of the Renoise LFO/meta devices, and various pattern effects. By this xStream could be abused to make LFO/meta devices control pattern data in a more familiar fashion. For instance controlling pitch (with pitch effects) via an LFO meta device. This idea would work with realtime streaming only, though.

Don’t forget that the output of xStream always is written (one or more lines) ahead of time. Also, if you change an arguments which require new output, those lines will be re-computed at a later time.
So, relying on a otherwise stable LFO movement might not behave as you’d expect it to.

Hey there, first of all xStream is amazing!
Turns out I’m a total noob when it comes to programming though so I’d need some help, if you don’t mind me asking.

What I’m trying to do is playing a note every x lines across the whole song, so basically a counter I guess.
I’ve been trying to get this to work for days now to no avail.

If this is the wrong place to ask, I’m sorry.

What I’m trying to do is playing a note every x lines across the whole song, so basically a counter I guess.

I’ve been trying to get this to work for days now to no avail.

Have you seen the demo models that come with xStream? One of them is called “Demo-Periodic Output” and does exactly this …

The principle is simple enough: you have a global property called “xinc”, which is an ever-increasing value (counting from 1 and upwards, as long as you’re playing).

If you then use a modulo operator (%) you can do ‘something’ every Nth line.

if (xinc%4 ==0) then
 -- do something every 4th line
else
 -- do something for other lines
end

The model then adds offset and a few other things on top, but the basic logic really isthis simple.

Thanks for the quick reply!

I should have been more specific, I’m sorry.

What I’m actually trying to do is, say, play a note every second line. Then after 30 lines play a note every line for 10 lines. Rinse and repeat.

(I hope that sounds plausible.)

Here’s some actual code I’ve tried:

local trig
local length

if xinc%16 == 0 then
 trig = 1
 length = xinc + 4
elseif xinc == length then
 trig = 0
else
 trig = 0
end

if trig == 0 then
 if xinc%2 == 0 then
  xline.note_columns[3] = {
   note_value = 29,
   instrument_value = args.instr_idx3,
   volume_value = 40 + math.random(-10, 30)
  }
 else
  xline.note_columns[3] = {}
 end
elseif trig == 1 then
 xline.note_columns[3] = {
   note_value = 29,
   instrument_value = args.instr_idx3,
   volume_value = 30 + math.random(-10, 20)
  }
end

The code for writing the notes works in a specific xpos.line range, that’s close to what I want to do but not quite there.

The problem seems to be with storing the value for length, and that’s where I’m stuck.

Sorry to keep bugging you about this.

The problem seems to be with storing the value for length, and that’s where I’m stuck.

Ah, as long as you’re declaring a variable as local it will be lost on each iteration.

But you could declare it like this (doing a check for it’s existence beforehand):

if not trig then
 trig = 0
end
if not length then
 length = 0
end

if xinc%16 == 0 then
  trig = 1
  length = xinc + 4
elseif xinc == length then
  trig = 0
end

Notice that I remove the last “else trig = 0” statement, or trig would be set to 1 only when equal to the length.

The logic above makes trig equal to 1 for the first four lines, followed by 12 lines where it’s zero. Rinse and repeat :smiley:

Alternatively, you could declare trig and length as “userdata” and you’d be able to access the values first checking for their existence. This works because variables specified as userdata are initialized once, when the model is first loaded.

Hint: click the + sign below the code editor and choose ‘userdata’.

Userdata becomes especially handy when you have some complicated logic that you want to tie into a function of it’s own.

Ah, that explains a lot. Thanks so much for your help!

There’s a bit of weirdness around the user data folder.

First is that you have to restart renoise for it to take effect.

Second is if you don’t have a subfolder called “models”, it will cause this error when trying to create a new model.

6990 Screen Shot 2016-09-14 at 10.57.15 PM.png

So this seems to be a bug: the model doesn’t work if the last line is a comment.

This model works:

-- Use this as a template for your own creations.
column = xline.note_columns[1]
column.note_string = "C-4"

This model doesn’t work:

-- Use this as a template for your own creations.
column = xline.note_columns[1]
column.note_string = "C-4"
-- break!

Maybe I’m dense but that seems like a bug.

I’m struggling to understand how args work. I’m trying to change a value over time. I’m outputting a note value along with a volume, and decrementing the volume with each iteration. I’m using offline mode, “apply to selected track.” Here’s my code:

column = xline.note_columns[1]
column.note_string = "C-4"
column.instrument_string = "00"
column.volume_value = args.velocity
if args.velocity > 10 then
  args.velocity = args.velocity - 1
end

Starting with a velocity value of 7F (hex), I would expect it to write one line with a velocity of 7F, the next with 7E, and so on until it flatlines at 0A. Instead, it’s writing only one line – the very last line of the pattern – with a velocity of 40. This is a 40 line pattern.

If I take out the if statement, it writes to every single line.

I thought perhaps you can’t reassign an args value, but the ChordMemory model assigns new values to args.note_order, line_space, human_vol… so it seems possible.

I saw an earlier post about using a variable inside the model, so I gave this a shot:

if not vel then
  vel = args.velocity
end

column = xline.note_columns[1]
column.note_string = "C-4"
column.instrument_string = "00"
column.volume_value = vel
if vel > 10 then
  vel = vel - 1
end

This actually does work, the first time around… I would expect the vel value to be reassigned each time the model is run (aka each time I press the “apply to selected track” button). But it doesn’t – it retains its value each time around.

But you know what? I think I just worked it out :slight_smile: What I’m calling “first run” is synonmous with xinc == 0, right? Giving me:

if xinc == 0 then
  vel = args.velocity
end

column = xline.note_columns[1]
column.note_string = "C-4"
column.instrument_string = "00"
column.volume_value = vel
if vel > 10 then
  vel = vel - 1
end

Which does the trick.

(I’m leaving this here in case anyone finds it helpful. Also, I’m confused as to why the original args.velocity approach didn’t work… although generally I prefer this anyway since I can manipulate the args value independently of running the model)