Jump to content


Photo

New Tool (3.0,3.1): xStream

live coding sandbox

  • Please log in to reply
131 replies to this topic

#1 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6359 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 26 September 2015 - 13:49

splash-big-mkII.png

 

Introducing xStream
xStream allows you to generate or transform existing pattern/automation data via custom code which can be created and tweaked in realtime

You can obtain the tool from the tools page, or github

 

Documentation 
When you install the tool, a 'README.md' is located in the installation folder. This is a markdown-formatted documentation file, including a quick reference to the xStream-specific Lua API. You can also visit the xStream github page for the newest version of documentation, regardless of what version you have installed. 
 
And of course, you can study the example models. For newcomers, it is highly recommended to start with the models which are prefixed with 'Demo-'. These models are designed to demonstrate a particular aspect of xStream without being overly complex. 
 
 


Edited by danoise, 23 September 2016 - 19:32.

  • jiku, Ledger, TheBellows and 8 others like this

Tracking with Stuff. API wishlist | Soundcloud


#2 afta8

afta8

    Big Super GrandMasta Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPip
  • 709 posts
  • Gender:Male
  • Location:London

Posted 26 September 2015 - 14:56

Amazing!

 

Hope someone films and uploads your demo of this tonight


  • Akiz likes this

#3 Fsus4

Fsus4

    Super Advanced Member

  • Normal Members
  • PipPipPipPip
  • 150 posts

Posted 27 September 2015 - 00:33

Hello danoise

 

I'm normally just an occasional lurker at these forums, but this particular tool of yours forces me to sign in and comment. :)

 

First of all, seems like an interesting concept. I’ve only tested the tool for half an hour, but I'm looking forward to explore its full potential.

 

Now, one of the ideas for a macro I immediately felt the need to create was a custom chord generator. Unfortunately I don’t speak this lua language, but I managed to figure this much out by copy-pasting from your examples into the empty slot.

--Creates a minor chord from the note value in note column 1
--
--Example:

--enter C-4 (or any other note value) in note column 1 in a track

--select the row

--press fill_selection button in xStream methods
--the C-4 (or other note value) will be replaced with C-4 +
-- D#4 + G-4 which is the C minor chord
--
--------------------------------------------------------------
 
local existing_note = xline.note_columns[1].note_value
 
xline.note_columns = {
  {
    note_value = existing_note + 0,
    instrument_value = 13,
    volume_value = args.volume,
    panning_value = 80
  },
  {
    note_value = existing_note +3,
    instrument_value = 13,
    volume_value = args.volume
  },
  {
    note_value = existing_note +7,
    instrument_value = 13,
    volume_value = args.volume
  }
}

Now if I want to manipulate the Cx (cutoff note) parameter in the panning_value so that it is for example “C8”, what needs to be done? It only seems to accept numerical values. Any suggestions?


Edited by Fsus4, 27 September 2015 - 01:10.


#4 MurderatØr

MurderatØr

    Big Masta Member

  • Banned
  • PipPipPipPipPipPipPipPipPip
  • 528 posts
  • Gender:Male
  • Location:UK
  • Interests:Let's not MENTION ZE WAR! You MUST like Renoise, Ve hev vayz of making you comply. o/

Posted 27 September 2015 - 03:09

Hi,

 

How about an example workflow for noobs that just want to make music, although they are grateful for the tool ... :-S


I mean, I'm getting results but I've no idea what it's doing.


My impact on the industry (one day): https://youtu.be/pTifdoKXoxM?t=1

#5 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6359 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 27 September 2015 - 15:20

Hey fsus, thanks -

it's really encouraging to know that you could use the tool without much prior knowledge 

 

Now if I want to manipulate the Cx (cutoff note) parameter in the panning_value so that it is for example “C8”, what needs to be done? It only seems to accept numerical values. Any suggestions?

 

It follows the Renoise API way of specifying things. So, you can use both value or string, whichever is the most handy. 

-- here, the value and string amount to the same thing - a C-8 note
note_string = "C-8"
note_value = 96 

The API documentation has the full information on note and effect-column properties, including special ones like "OFF" 


Tracking with Stuff. API wishlist | Soundcloud


#6 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6359 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 27 September 2015 - 15:29

How about an example workflow for noobs

 

Hm... one example would be to select the "Notes and scales" model and then select some pattern data and hit 'Fill selection'

--> You have just restricted all notes to that particular scale/key

--> This is also what I refer to as the offline mode

 

Another one (my favourite) would be to select "Random increase", disable 'clear_undefined', and let playback enter a blockloop

--> Now you will hear an endlessly evolving/repeating pattern of notes. Try switching instrument while playing :-D

--> Setting 'clear_undefined' to false means that we add new notes on top of existing ones

--> This is also what I refer to as online, or 'streaming' mode

 

Whether you find that useful or not, is a different matter  ^_^


Tracking with Stuff. API wishlist | Soundcloud


#7 Fsus4

Fsus4

    Super Advanced Member

  • Normal Members
  • PipPipPipPip
  • 150 posts

Posted 27 September 2015 - 16:53

Aha, reading from that API stuff I figured out that one needs to write this

 

panning_value = 0x00000C08

 

in order to place out a C8 (Cx: Cut note after X ticks (0 - TPL)) in the panning column

 

Lovely, I need to check this lua stuff out.

 

Hey fsus, thanks -

it's really encouraging to know that you could use the tool without much prior knowledge 

 

 

It follows the Renoise API way of specifying things. So, you can use both value or string, whichever is the most handy. 

-- here, the value and string amount to the same thing - a C-8 note
note_string = "C-8"
note_value = 96 

The API documentation has the full information on note and effect-column properties, including special ones like "OFF" 



#8 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6359 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 27 September 2015 - 17:36

True, you can specify any possible value using numbers. It can be a little complex though, because you'd need to pass values through a little calculation first. 

In most cases, strings are much easier to work with, what you see in the pattern editor you can also enter as a string value:

panning_string = "C8"

If the value "8" happened to be the result of some kind of calculation, you could hand over a string by using string.format() 

local my_ticks_value = 8
panning_string = ("C%d"):format(my_ticks_value)

But if you are applying a value as a result of some kind of expression, or you are comparing values, numbers are useful. 

For example, this little snippet will only transform notes that are *not* empty or note-offs

-- transpose notes down one semitone
if (xline.note_columns[1].note_value < 120) then
  xline.note_columns[1].note_value = xline.note_columns[1].note_value - 1
end

This is so, because the "note off" and "empty" are both higher than 119 (120 and 121, respectively)


Tracking with Stuff. API wishlist | Soundcloud


#9 MurderatØr

MurderatØr

    Big Masta Member

  • Banned
  • PipPipPipPipPipPipPipPipPip
  • 528 posts
  • Gender:Male
  • Location:UK
  • Interests:Let's not MENTION ZE WAR! You MUST like Renoise, Ve hev vayz of making you comply. o/

Posted 27 September 2015 - 18:05

Hm... one example would be to select the "Notes and scales" model and then select some pattern data and hit 'Fill selection'

--> You have just restricted all notes to that particular scale/key

--> This is also what I refer to as the offline mode

 

Another one (my favourite) would be to select "Random increase", disable 'clear_undefined', and let playback enter a blockloop

--> Now you will hear an endlessly evolving/repeating pattern of notes. Try switching instrument while playing :-D

--> Setting 'clear_undefined' to false means that we add new notes on top of existing ones

--> This is also what I refer to as online, or 'streaming' mode

 

Whether you find that useful or not, is a different matter  ^_^

 

 

Hi,

 

For the first suggestion, I just get the same 3 notes played in the same sequence, endlessly, throughout the track.

 

For the second suggestion, I keep getting a weird crescendo, going on and on, in the same track (even when i select a wide range of tracks).

 

Perhaps a video is needed, or a manual :)


From what I can see, this would be good for generating riffs, but not live performance as such. I think RPG and a 2-out soundcard can beat this, based on the limited experience I've had so far with xStream.

 

 

EDIT: I think I see the potential of this, if a person knows coding, they can make it do anything ...


Edited by MurderatØr, 27 September 2015 - 22:45.

My impact on the industry (one day): https://youtu.be/pTifdoKXoxM?t=1

#10 Fsus4

Fsus4

    Super Advanced Member

  • Normal Members
  • PipPipPipPip
  • 150 posts

Posted 27 September 2015 - 19:44

For me xStream seems to have a lot more potential in offline mode.

 

I imagine in the future xStream would host a huge number of models/buttons (placed on a second screen in dual monitor setup), each button executing an unique custom macro operation. Maybe hundreds of such models would need to be placed within selectable categories or something for a better overview. This would be my first feature request, because it would make it easier to organize stuff.

 

Anyway, this doesn't work for me:

local my_ticks_value = 8
xline.note_columns = {
  {
    note_value = 36,
    instrument_value = 7,
    volume_value = args.volume,
    panning_string = ("C%d"):format(my_ticks_value)
  }

All the values render properly, it just refuses to place out the 'C8' in the panning column. But maybe one can't mix _values with _strings?


Edited by Fsus4, 27 September 2015 - 19:47.


#11 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6359 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 28 September 2015 - 12:36

I imagine in the future xStream would host a huge number of models/buttons (placed on a second screen in dual monitor setup), each button executing an unique custom macro operation. Maybe hundreds of such models would need to be placed within selectable categories or something for a better overview. This would be my first feature request, because it would make it easier to organize stuff.

Indeed, this is something I've been thinking about as well. Right now, it's presenting whatever models you have in a single folder - logical next steps would be to list those models in a table, grid or some other container, along with the ability to categorize presets by putting them into subfolder (I'm a big fan of using the file system to describe a hierarchal structure). Right now, it's a problem that you so easily can run out of space on the screen.

And while "hundreds of models" might sound a little overwhelming, yeah - why not. I think I would personally aim for a little more flexibility in my models, and use the preset system to create unique variations thereof - but then, obviously the preset system too would need the ability to host hundreds of entries :-D
 

Anyway, this doesn't work for me:

Well spotted! There was indeed a little mistake when applying alfa-numeric values that could be interpreted as hex (U8 would work, but your C8 wouldn't.. uploading a fix ASAP

For the first suggestion, I just get the same 3 notes played in the same sequence, endlessly, throughout the track.

Well, it depends on what you "feed" it - since that model is transforming existing pattern data, the notes will remain the same. If you had some melody line, chord sequence, you should be able to hear a difference.
  • Fsus4 likes this

Tracking with Stuff. API wishlist | Soundcloud


#12 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6359 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 28 September 2015 - 12:57

For me xStream seems to have a lot more potential in offline mode.

 
This is also a point which was raised at the meetup. Basically, xStream will, in its current shape, allow you to do "only one thing at a time". 
While that of course makes sense in this early stage, it also is quite limiting from a live performance, jamming point of view. 
 
My longer-term ideas for xStream involves having a kind of "stream manager", essentially a type of arranger which will allow you to control multiple models, and schedule events at certain times. 
But right now, this is really a pipe dream because of the amount of work involved. Focusing on making the core as efficient and stable as possible has the top priority!!
  • Fsus4 likes this

Tracking with Stuff. API wishlist | Soundcloud


#13 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6359 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 28 September 2015 - 22:40

Version 1.01 is on github now

FIXME attempt to fix crash-proneness : apply the callback string during idle loop instead of right away
FEATURE detect when model is changed, ability to revert to last saved state
CORE better unit tests for song pos

Edit: this update seems to resolve one of the major problems so far, which was a tendency to crash Renoise - especially true with the most complex models ('Sequencer4').

My theory as to why this could happen? It turned out that, when switching between models, or setting a model for the first time, the act of updating the user interface - more specifically, the textfield containing the callback code - would call it's notifier *once for every line*. This would result in xStream trying to compile the callback method maybe 50 times in a split-second...

While doing such a thing shouldn't really be able to take down Renoise - ever - xStream is using some pretty "unusual" features of the lua language, and as such, we might have headed a little bit into unknown/untested territory here.

Bottom line is, I have improved xStream in this regard - the user interface will now wait for the idle update before submitting a modified string. Hopefully, this should prevent crashes from happening. Please report this if it still happens to you.

Also, there is now a "revert to saved" button, and the "save" button is enabled only when the model has been modified.


Edited by danoise, 29 September 2015 - 09:50.

Tracking with Stuff. API wishlist | Soundcloud


#14 taktik

taktik

    Renoise Developer

  • Admins
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 15040 posts
  • Gender:Male
  • Location:Berlin, Germany
  • Interests:füße waschen

Posted 29 September 2015 - 12:05

Version 1.01 is on github now

FIXME attempt to fix crash-proneness : apply the callback string during idle loop instead of right away
FEATURE detect when model is changed, ability to revert to last saved state
CORE better unit tests for song pos

Edit: this update seems to resolve one of the major problems so far, which was a tendency to crash Renoise - especially true with the most complex models ('Sequencer4').

 

Have been working on that too and found out what the problem is: The buttons which do select new models do update/rebuilt all buttons which select new models too when clicking them, so pressing a model button killed the button that was currently pressed. We're internally avoiding doing such things, but for tools we can't make such "prerequisites", so I'll fix that here for the next Renoise update.

 

If you need a workaround for your tool now: avoid rebuilding the model buttons when selecting a new model via a model button - then it should work.


  • danoise, crazya02 and Fsus4 like this

#15 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6359 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 29 September 2015 - 16:40

1.02 is available now on github and the tools page

FIXME Avoid recreating model/preset button-lists as a result of these buttons being clicked

Tracking with Stuff. API wishlist | Soundcloud


#16 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6359 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 30 September 2015 - 18:02

Uploading a full hour (!!) of video right now (check back later)

 

It's my presentation of the xStream tool for Renoise at the meetup last Saturday. 

  • Djeroek, ffx, Akiz and 4 others like this

Tracking with Stuff. API wishlist | Soundcloud


#17 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6359 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 01 October 2015 - 15:56

Regarding that video above: If someone reaches the one hour point, they are rewarded with something reminiscent of actual music. 

 

Also, it could be turned into a drinking game:

If you take a hit every time I say "the thing is...", you will be blasted quite quickly  :smashed:


  • radian and pat like this

Tracking with Stuff. API wishlist | Soundcloud


#18 ffx

ffx

    Guruh Motha Fakka is Levitating and Knows Everything About Renoise Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • 2897 posts
  • Gender:Not Telling
  • Interests:Renoise sidechaining, macOS fanboying

Posted 01 October 2015 - 16:23

TBH the audio in video needs some compression for upper volumes, expandation/gating for quitter stuff and eq on highs. Was quite difficult at least for me to understand the words...  My respect you did this presentation and explained it so detailed.


Edited by Jurek, 01 October 2015 - 16:24.

MacOS 10.12.6 Retina, Renoise 3.1 64 bit   -   Tuned Shortcuts | Multi-Jump From/To Send | Quick Template | Insert Native DSP Menu (incl. deprecated)


#19 Akiz

Akiz

    Guruh Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPip
  • 967 posts
  • Gender:Male

Posted 01 October 2015 - 17:52

3.1 spoiler, nice :)


  • ffx likes this

#20 ffx

ffx

    Guruh Motha Fakka is Levitating and Knows Everything About Renoise Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • 2897 posts
  • Gender:Not Telling
  • Interests:Renoise sidechaining, macOS fanboying

Posted 02 October 2015 - 18:13

Erm I have have some questions regarding making a algorithm with it...

 

Here is the code:

See post below

 

My questions are:

 

1. How do I add a slider value?   "reveal_location" just seems to unfocus renoise.  

 

2. In the example above, if num_notes are > 3, the iterator n suddenly starts with 3 instead 0... Do I made a mistake? Don't get it...

 

3. How can I check for a note-off, so it would be detected a an note?

 

4. will the note-offs automatically moved?

 

Thanks for help!


Edited by Jurek, 03 October 2015 - 20:30.

MacOS 10.12.6 Retina, Renoise 3.1 64 bit   -   Tuned Shortcuts | Multi-Jump From/To Send | Quick Template | Insert Native DSP Menu (incl. deprecated)


#21 Roppenzo

Roppenzo

    Big Daddy Member

  • Normal Members
  • PipPipPipPipPipPipPip
  • 441 posts
  • Gender:Male
  • Location:Netherlands

Posted 03 October 2015 - 15:58

Thanks for the presentation vid danoise :-)
  • redpanda13 likes this
House Music All Night Long. - Say What?

#22 Djeroek

Djeroek

    Probably More God or Borg Than Human Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6706 posts
  • Gender:Male
  • Location:Borneo

Posted 03 October 2015 - 16:57

Can this tool go through a selection of notes and sort these over time, from lowest note to highest note or the other way around? The time it takes to sort, generated as pattern content...something like this;


  • afta8 and ffx like this

#23 ffx

ffx

    Guruh Motha Fakka is Levitating and Knows Everything About Renoise Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • 2897 posts
  • Gender:Not Telling
  • Interests:Renoise sidechaining, macOS fanboying

Posted 03 October 2015 - 17:36

cool stuff


  • redpanda13 likes this

MacOS 10.12.6 Retina, Renoise 3.1 64 bit   -   Tuned Shortcuts | Multi-Jump From/To Send | Quick Template | Insert Native DSP Menu (incl. deprecated)


#24 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6359 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 03 October 2015 - 17:54

Erm I have have some questions regarding making a algorithm with it...

 

1. To add a slider (or popup, checkbox and so on) to control arguments you need to define it in the model definition (lua file)

This is most easily done by hitting 'reveal location', it _should_ open explorer, finder etc. and focus the lua file. If not (running OSX?) then I need to look into it. 

 

But the principle is the same - open the lua file in a text editor of some kind, and perhaps copy-paste one of the arguments from another definition.

I have not fully documented the arguments and their properties, all the possibilities are currently only available 'by example'

 

2. You seem to be initializing the variable 'n' the first time the callback is executed, then adding on consecutive runs. Eventually it will reach the value 3. 

But the easiest would probably be to explain the purpose of your code.

 

Edit: maybe you just want to 'push' note-column 1 into note-column 2? This could expressed as simple as this:

xline.note_columns[2] = xline.note_columns[1]
xline.note_columns[1] = {} 

3. You can detect a note-off either by it's value or string: 

if (note_value == 120) then
 -- note off detected
end

if (note_string == "OFF") then
 -- note off detected
end

xStream is using the same syntax as in the Renoise API for note and effect columns: 

https://github.com/r...g.API.lua#L2805

 

4. Depends on what you do with them. Note-offs (or even empty notes) is not a 'special case', they are merely values/strings.

The only special case would be if you try to transpose a note beyond it's valid range. C-0 is the lowest possible note, so

transposing such a note down would make it remain at C-0 and log a warning message in the scripting console :-)

 

--

 

Btw: as I wanted to test-drive your code, I hit the 'new' button to create a new model from scratch.

But this actually threw an error, seems to be a small bug which was introduced in v1.02.

 

Will fix, of course. 


Edited by danoise, 03 October 2015 - 19:05.

  • redpanda13 likes this

Tracking with Stuff. API wishlist | Soundcloud


#25 ffx

ffx

    Guruh Motha Fakka is Levitating and Knows Everything About Renoise Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • 2897 posts
  • Gender:Not Telling
  • Interests:Renoise sidechaining, macOS fanboying

Posted 03 October 2015 - 19:05

Thank you for the infos!

 

by hitting 'reveal location', it _should_ open explorer, finder etc. and focus the lua file. If not (running OSX?) then I need to look into it. 

 

Yes, doesn't seem to work on OSX so far... Edited the definition, works now.  The only strange thing is, it seems I have to completely disable and re-enable xstream, before I can see any external made changes.

 

 

2. You seem to be initializing the variable 'n' the first time the callback is executed, then adding on consecutive runs. Eventually it will reach the value 3. 

But the easiest would probably be to explain the purpose of your code. I think you want to 'push' note-column 1 into note-column 2? 

 

I wanna do some note splitting, I the way that for each num_notes on one col, it will be splitted to num_tracks/cols. E.g. having a mono 16th note track, settings 1,2, it should split the notes alternating.  I want to do this for multiple vsti instancing, individual note panning.

 

I don't want to use this in live. Only as an editing tool.

 

So here I would like to have a counter that initializes every time I press "fill track" on row 1. So currently, it won't initialize on repeating press on "fill track"?  

 

You can detect a note-off either by it's value or string: 

Depends on what you do with them. Note-offs (or even empty notes) is not a 'special case', they are merely values/strings.

 

Thanks, I made some logical mistake, now it seem to work to move the note-off along with the note.

 

 

Spoiler

Edited by Jurek, 03 October 2015 - 20:29.

MacOS 10.12.6 Retina, Renoise 3.1 64 bit   -   Tuned Shortcuts | Multi-Jump From/To Send | Quick Template | Insert Native DSP Menu (incl. deprecated)






Also tagged with one or more of these keywords: live coding, sandbox