Jump to content


Photo

New Tool (3.0,3.1): xStream

live coding sandbox

  • Please log in to reply
183 replies to this topic

#151 joule

joule

    Guruh Motha Fakka is Levitating and Knows Everything About Renoise Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • 1520 posts
  • Gender:Not Telling
  • Location:Sweden
  • Interests:music, philosophy, engineering

Posted 18 October 2017 - 13:27

Sounds very promising. xLine is at the heart of xStream - it could indeed bring a lot of extra performance  :)

 

1) I assume that 90% of overhead is related to song() access and 10% to the classes/flexibility :) It seems that this can be optimized simply by modifying xLine.do_read and xLine.do_write, if I want to experiment?

 

2) Is your buffering-system a bottleneck in itself (using idle loop), or will the idle loop make the performance adapt? What I'm asking is if speed improvements will be noticable when stress testing, or if the 0.1s update frequency will work as some sort of constant, with the only variable being how many lines ahead are needed for reliable writing?

 

EDIT: Well, I can use the TRK button for measurements anyway.


Edited by joule, 18 October 2017 - 13:30.


#152 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6492 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 18 October 2017 - 14:33

xLine.do_read and xLine.do_write, if I want to experiment?

 

Yes - but not quite that simple. If the xLine itself managed everything - note columns, effect columns and automation - it would be quite a monster class.

Instead, you want to look closer at xLinePattern (which really should be called xPatternLine, eheh...). This class deals with things that can be expressed through the pattern (basically the xLine without the automation envelope component). 

And if you keep following the breadcrumb trail, it will take you to xNoteColumn and xEffectColumn as well  :)

 

Together, those classes form a complete, 'virtual' representation of pattern data, without any references to renoise.song().

The purpose of the xNote/EffectColumn classes is also to accept both numbers and strings as input (number_string/amount_string), which is always stored internally in the class as a number (number_value, amount_value).

But with the optimization you have in mind, it might make sense to flip this around and make those classes prefer strings internally. Otherwise, there will be some unnecessary converting back and forth. 

 

Btw: I have just committed a few things on github to properly split the pending changes for the stacked model. 

The goal is simply for us to have a clean slate to work on. So master is now the current version of xStream (1.57) + some bugfixes I've committed in the meantime. 

I'm testing this a bit right now, to make sure I didn't accidentally break something...


Tracking with Stuff. API wishlist | Soundcloud


#153 joule

joule

    Guruh Motha Fakka is Levitating and Knows Everything About Renoise Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • 1520 posts
  • Gender:Not Telling
  • Location:Sweden
  • Interests:music, philosophy, engineering

Posted 18 October 2017 - 16:03

I'm familiarizing myself with the structure.. I guess it would be fine to just hi-jack the xLinePattern.do_read() to start with.

 

By the way! When TRK-rendering a track with only one note column and 256 lines, the do_read() function is triggered 32896 times (checked with a global variable++). I also get the feeling that the number of times this function is executed is non-linear to the number of lines (136 times triggered on a 16 line pattern). I'm mentioning it if something really fishy is going on that takes up a lot of resources.



#154 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6492 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 18 October 2017 - 16:35

I'm familiarizing myself with the structure.. I guess it would be fine to just hi-jack the xLinePattern.do_read() to start with

 

Sure, working on a single part should make it easier for both of us. 

 

 

By the way! When TRK-rendering a track with only one note column and 256 lines, the do_read() function is triggered 32896 times (checked with a global variable++).

 

Haha, that's a lot !!

But yes, there would be an overhead when streaming, because it reads lines multiple times to pick up "just in time" changes. 

But it shouldn't be *that* much, and not needed at all when applying to track ("offline mode"). So, yes, something's stinky in there. 

 

To be fair, I have deep-dived into the "xStreamBuffer" in the newer sources (the stacked model branch). So that one is already a lot more efficient in the whole input/output department. 

Guess I'll focus on that branch, and figure out how it can be merged nicely what you're bringing to the (lua) table  ;)


Tracking with Stuff. API wishlist | Soundcloud


#155 joule

joule

    Guruh Motha Fakka is Levitating and Knows Everything About Renoise Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • 1520 posts
  • Gender:Not Telling
  • Location:Sweden
  • Interests:music, philosophy, engineering

Posted 18 October 2017 - 20:11

I think I managed so commit something on Github. It turns out the overhead of other stuff is a lot more than i expected, so the speed-up of do_read() is only 66% or so (depending on how many columns that are read).


Edited by joule, 18 October 2017 - 20:11.

  • danoise likes this

#156 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6492 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 18 October 2017 - 21:14

Merged !! 

66% faster is what I call a substantial speed-up. And nice and readable, too :-D

 

If you were expecting more, well, blame it on me. I will port some of my improvements over in the following days - 

that should prove especially beneficial for the offline mode ;)


Tracking with Stuff. API wishlist | Soundcloud


#157 boonier

boonier

    Advanced Member

  • Normal Members
  • PipPipPip
  • 116 posts

Posted 18 October 2017 - 23:17

Yep. xStream is currently hard-coded to the selected track. Again, this is something that would change in the planned version 

(I would like to add as a sidenote: it's nice how your questions align so nicely with those plans  ^_^ )

 

Hehe. Just fresh eyes man, fresh eyes!



#158 joule

joule

    Guruh Motha Fakka is Levitating and Knows Everything About Renoise Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • 1520 posts
  • Gender:Not Telling
  • Location:Sweden
  • Interests:music, philosophy, engineering

Posted 19 October 2017 - 07:22

Together, those classes form a complete, 'virtual' representation of pattern data, without any references to renoise.song().

The purpose of the xNote/EffectColumn classes is also to accept both numbers and strings as input (number_string/amount_string), which is always stored internally in the class as a number (number_value, amount_value).

But with the optimization you have in mind, it might make sense to flip this around and make those classes prefer strings internally. Otherwise, there will be some unnecessary converting back and forth.

 

I think it makes sense to use strings only, as Renoise accepts them. Potentially, values could be provided on a need-to-use basis with a minimum amount of conversion needed. I think the conversion functions took about 50% of the total time or sth, when parsing a patternstring :(

 

By the way. I'm hitting the  "*** main.lua:155: attempt to index field 'preferences' (a nil value)" error on one computer with my update. I remember you said it had to do with how many variables that are used. This must be per file, and not just in the global scope? I'm now at a sweet spot where I can turn the error on and off by typing "local test" inside of a function.

 

EDIT: According to the Corona forum, the maximum amount of local variables per file in LUA is 200. It seems like a good idea to table-ize variables that aren't accessed very often.


Edited by joule, 19 October 2017 - 08:31.


#159 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6492 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 19 October 2017 - 09:43

I have created a branch containing the newer buffer implementation, which, as suspected, makes things a lot more efficient:

https://github.com/r...-implementation

(still needs some testing, but good enough as a proof-of-concept)

 

You can see for yourself, as I have added a small console print to both the master and this branch that tells you how long time it takes to process a track. 

 

The difference is most striking in offline mode, where it can bring more than 20x times the speed.

 

Euclidean Rhythms 1.2 ("whispers" preset)

Before: ~8 seconds

After: ~0.3 seconds

 

That's an improvement of, well, embarrassing dimensions !! 

 

Had I known it would've made such an impact, I would surely have back-ported this feature when I originally wrote it four months ago.

But, I also think think I know why it escaped me: the tool is tested on my netbook - my current "lower computing threshold". 

And since realtime streaming was always working fine on that machine, I've not really experienced just *how* much of a difference the newer implementation made. 

 

Moral of the story: profiling code is boring important. 

 

 

By the way. I'm hitting the  "*** main.lua:155: attempt to index field 'preferences' (a nil value)" error on one computer with my update. I remember you said it had to do with how many variables that are used. 

 

You're saying it happens even with the current/master? In that case, ugh... I have only seem it happen when the tool is initializing with the full set of keyboard/midi mappings.

 

EDIT: According to the Corona forum, the maximum amount of local variables per file in LUA is 200. It seems like a good idea to table-ize variables that aren't accessed very often.

 

That local-variable limit can be set as a compile time property, so it seems Corona are compiling with 200 variables. I think we accept 300. 

 

But unfortunately that isn't the problem here - there are no variables being declared inside those mapping definitions.

Whether the problem occurs or not seems to be a combination of the code and the computer it's running on - it really is quite mysterious. 


  • joule and pat like this

Tracking with Stuff. API wishlist | Soundcloud


#160 joule

joule

    Guruh Motha Fakka is Levitating and Knows Everything About Renoise Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • 1520 posts
  • Gender:Not Telling
  • Location:Sweden
  • Interests:music, philosophy, engineering

Posted 20 October 2017 - 09:15

Good stuff! I think it's working ok. The rendering time also seems linear to amount of lines rendered. (But the optimization i submitted doesn't seem to make much of a difference, so I'm guessing it's other stuff that still takes time). There seemed to be a lot of other bugs in that branch, but I'm guessing that you're sorting it all out for an update (perhaps also a case of me not having exactly the same vlib/clib/xlib as you).



#161 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6492 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 20 October 2017 - 10:09

@joule: you forgot to mention if you got the "preferences (nil)" issue on this test branch or some code of your own? 

 

perhaps also a case of me not having exactly the same vlib/clib/xlib as you

 

I'm having this slightly ghetto solution where I symlink the libraries into the tool folder. 

This ensures that, whatever tool I'm working on, the changes are propagated to other tools that use those libraries too. 

 

So to make things work on your side, you'd have to either copy or symlink those libraries - from the respective folders. 

For example: 

 

com.renoise.xStream.xrnx/source/xLib => symlink => com.renoise.xLib.xrnx/source/xLib

 

The .gitignore should make sure that the symlinked folders are not cleaned when switching branches. 

This means that the symlinks themselves manage to 'survive' such git actions and still point to the right place. 

 

When it works, it actually works quite well  :)


Tracking with Stuff. API wishlist | Soundcloud


#162 joule

joule

    Guruh Motha Fakka is Levitating and Knows Everything About Renoise Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • 1520 posts
  • Gender:Not Telling
  • Location:Sweden
  • Interests:music, philosophy, engineering

Posted 20 October 2017 - 10:29

The preferences (nil) warning seems to be gone! I was having a look at the do_write function. If I understand correctly, you only write to .song() those columns that are updated (and quickly skip/clear empty lines when target is already empty?), so it should be pretty optimized already.

 

PS. I made a line copy function that is even a lot faster than the native :copy_from(), but it's only usable if you're certain enough that the amount of unnecessary column writes are more than the one tostring(line) access that is required. I think I'll post a simple class with these functions in another thread, as food for thought.


Edited by joule, 20 October 2017 - 10:34.


#163 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6492 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 20 October 2017 - 10:42

Btw: you also mentioned you had encountered some bugs? 

 

I'm a big fan of using the github issue tracker.

If you click the following link, you will see that they are also organized in milestones (xStream 1.6 and 2.0, at time of writing): 

 

https://github.com/r...is:open xstream


Tracking with Stuff. API wishlist | Soundcloud


#164 g8tr1522

g8tr1522

    Member

  • Normal Members
  • PipPip
  • 25 posts
  • Gender:Male
  • Location:Gainesville, FL

Posted 06 November 2017 - 15:59

Yep. xStream is currently hard-coded to the selected track. Again, this is something that would change in the planned version 

(I would like to add as a sidenote: it's nice how your questions align so nicely with those plans  ^_^ )

 

How much longer until this plan comes to action? I need a multitrack xStream man!!!  :(  But seriously, I had a ton of fun with it this weekend, but was annoyed at how my bass line AND my drum sequence couldn't be auto-composed simultaneously. My crappy suggestion for a temporary solution: allow a model to 'go in the background'. Like, it maintains control over (and continues editing the pattern data of) a certain track, but it is inaccessible to the user while they have another model open. So they can't edit the first model, or mess with the arguments GUI of that model.

 

Then, perhaps all running models would be on a track-by-track basis. One model per track. And when I select another track, it opens up the model assigned to that one. Not a big request at all, i know. :D 


In other problems-i've-been-running-into: How does one edit a model in the Renoise scripting console? 

 

However, I have been toying with an idea which would enable you to edit xstream files outside Renoise. I think this is actually a much more convenient thing, because no matter how much how we tweak the text editor in Renoise, most people who are dabbling with this tool probably have a text editor that they prefer anyway. So I see no reason why xStream shouldn't benefit from that. Hell, you could even use the Renoise scripting console then, if you wanted to (I know some people do ^_^)

If I open up my model in the console, it's in that sandbox section, which is a big block comment. Thus, there's no syntax highlighting. Not much of an improvement over the tiny black and white text in the xStream console. 

But this isn't even the worse issue. When I do save changes that I made in the Renoise scripting console, the current instance of xStream suddenly closes. I have to open it and my model back up to try and execute the new code. But instead of my new code being there, my model is instead a blank canvas. It's still inside the Renoise editor, but not in the xStream one. So I have to copy and paste it from the Renoise editor to the xStream one. 

 

I've noticed this sometimes only happens if my new code has a syntax error. If it doesn't, sometimes the model won't be blank when I open it back up. I'd be more than happy to go back and figure out/report on the exact behavior of all this if you'd like.

But not a big issue. Just making you aware. I'm perfectly fine with the xStream editor in the meantime. Also weird: if I make a change in the xStream editor (and save), it does not update those changes in the Renoise editor. Haven't tried with other editors yet. I think this also had that syntax error peculiarity. 

 

Second issue I've been having is that I get this error:

*** Error: please review the callback function - .\source/xLib/classes/xEffectColumn.lua:216: Unexpected effect number. Expected two bytes between 0-35 respectively  

It prints this constantly (in the renoise console) when I run this one model I've been using. But it's updating the effect columns just fine. I can send you the model and the song. Should I zip it?

 

Final issue, another weird error:
*** Expected an instance of xline for output - sequence:  4  line:  46  

I also get this persistently when running my models. Sometimes, I get it and no track data is written to. Not until I click and change some args (eg, click a 'choose' button, move a minislider, etc.) will it then start putting out track data. And sometimes the error then goes away. Sometimes it doesn't. 

 

One question: Is this worth pursuing? 

I want to use Ruby array methods in xStream. I was thinking I would get these methods to work in a C program by calling the Ruby method as an external function to the C program. Then, I would try to get a Lua program to call a C function, which then just calls the Ruby method. Would this be recommended in xStream? Any problems as far as performance issues?

 

If not, I'd like to get that working. Array manipulation is easy peesy in Sonic Pi because of all the different methods there were. I'd like that in xStream. Otherwise, I'll probably just write them myself. Would there be any performance issues if I wrote them in C and called the C functions in xStream?

 

 

Also, thanks again for this amazing tool. It's inspired me to start making music consistently. Before, I'd make something like, 3 times max a year. Now I'm pumping out another thing every other week. And they're all miles better than what I had made before.



#165 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6492 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 06 November 2017 - 19:20

How much longer until this plan comes to action?

 
My roadmap includes a new "stable" release shortly, without the multi-track/model feature, but featuring all the improvements that I and joule have discussed above.
And then, development will switch to github for a while, which means that you could try out early versions and provide a bit of feedback. 
 

How does one edit a model in the Renoise scripting console?

 

Yes, coding in that tiny tool window is a bit claustrophobic. But for now, it's really what we got.
But you've just convinced me that I should try sneaking this feature into the next release, because I also feel that pain:
http://forum.renoise...tream/?p=360863
 

Second issue I've been having is that I get this error:
*** Error: please review the callback function - .\source/xLib/classes/xEffectColumn.lua:216: Unexpected effect number. Expected two bytes between 0-35 respectively  
It prints this constantly (in the renoise console) when I run this one model I've been using. But it's updating the effect columns just fine. I can send you the model and the song. Should I zip it?
Final issue, another weird error:
*** Expected an instance of xline for output - sequence:  4  line:  46  
I also get this persistently when running my models. Sometimes, I get it and no track data is written to. Not until I click and change some args (eg, click a 'choose' button, move a minislider, etc.) will it then start putting out track data. And sometimes the error then goes away. Sometimes it doesn't.

(First one) yes please - zipped or non-zipped, you decide. 
(Second one) hmmm... not sure, but so many things can go wrong. It happens only with your own models? Then, PM please :-)
 

One question: I want to use Ruby array methods in xStream.

 

That would essentially be an extension of the Renoise API, if you wanted the performance too. Otherwise it will be really clunky and (probably) have quite an overhead.
But curious as to what those handy methods are - lua is completely table based, so quite powerful for arrays too.
And perhaps some of these methods are readily available elsewhere, as vanilly lua libraries? That would make things quite simple then.
 

It's inspired me to start making music consistently. Before, I'd make something like, 3 times max a year. Now I'm pumping out another thing every other week. And they're all miles better than what I had made before.

 

Hey - this is 10x better than a "thankyou" note. Really means a lot to me  :wub:


Tracking with Stuff. API wishlist | Soundcloud


#166 g8tr1522

g8tr1522

    Member

  • Normal Members
  • PipPip
  • 25 posts
  • Gender:Male
  • Location:Gainesville, FL

Posted 06 November 2017 - 19:29

Trying to get my own functions/libraries working right now. Correct me if I'm wrong about anything. Also, feel free to tell me to get out and ask my questions in the beginner's scripting forum if my questions are too easy/annoying.  :rolleyes:

.

Everything that gets included with xStream is specified in the main.lua  But - that doesn't mean that you can just starting referencing functions and method from within the xStream "main method" (the processing method that transforms pattern data, not to be confused with the "main.lua" file, which is simply the entry point for any renoise tool. 

Okay, just so we're on the same page, this means that I can't just do require statements like you did in main.lua?

 

 

 

Is there any way to include libraries? Do I just put a simple #include "lib.h" at the top of the code?

 

Everything that gets included with xStream is specified in the main.lua  But - that doesn't mean that you can just starting referencing functions and method from within the xStream "main method" (the processing method that transforms pattern data, not to be confused with the "main.lua" file, which is simply the entry point for any renoise tool. 

 

This is so, because xStream wraps the main method in a lua sandbox - which makes certain variables (e.g. the all-important xinc) available to the model, while also hiding potentially 'dangerous' things like file system access.  

In order to benefit from method in your own files, you need to explicitly define sandbox accessors to your own properties and methods. If it sounds complicated, it's not - just a question of opening and modifying the file called xStreamModel.lua.

 

So I gotta figure out this sandbox stuff I guess. 

 

I see the renoise scripting page for cSandbox.lua.  In it, it says that to supply functions to the sandbox, you do what it says to do in those examples at the very top. There's also a "to do" for how to include properties in a sandbox.

I'm mentioning this in the context of xStreamModel.lua. Seems like all the xStream user-end stuff that is 'given' to the sandbox is in that 'props_table' table. Then immediately following the definition of props_table (lines 127-309), you have the line self.sandbox.properties = props_table (line 311). I'm guessing that each of the member definitions in prop_table are supposed to make calls to the functions in "../source/xLib/classes" (and elsewhere)?
 

Should I just copy what you did there and make my own folder in "/source/.." with all my own properties? Seems messy (and I'm probably wrong).

 

 

Could I just add a singular property to props_table that (when called in xStream) points to a .lua file that will act as the foundation for my user library?

  • Eg, have a property "myLib" in props_table. Implement it just like the others.
  • In myLib.lua, I have require statements to more libraries (eg, lib1, lib2).
  • Then, I (somehow) make it where if I want to use a function in lib1, i just do "myLib.lib1.member_func" in the xStream console.

Or even better, can I skip the whole "editing xStreamModel.lua" by instead adding require statements to some Renoise file? :D  probably not... This just seems daunting to me. Not sure where to start. 


(First one) yes please - zipped or non-zipped, you decide. 

(Second one) hmmm... not sure, but so many things can go wrong. It happens only with your own models? Then, PM please :-)

 

Will definitely get to this later. Gotta go to class rn. Thanks again!



#167 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6492 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 06 November 2017 - 19:50

Could I just add a singular property to props_table that (when called in xStream) points to a .lua file that will act as the foundation for my user library?

  • Eg, have a property "myLib" in props_table. Implement it just like the others.
  • In myLib.lua, I have require statements to more libraries (eg, lib1, lib2).
  • Then, I (somehow) make it where if I want to use a function in lib1, i just do "myLib.lib1.member_func" in the xStream console.

Or even better, can I skip the whole "editing xStreamModel.lua" by instead adding require statements to some Renoise file? :D  

 

Yep (eh, or nope, depending on how you read it). xStreamModel is based on cSandbox, and as such, acting as the "gatekeeper". This is so, so some script kiddie can't just share a model which tries to wipe peoples harddrive. Copy-paste is handy, but there's the downside. You see, the tool itself has access to file system and such   ^_^

But adding your class should indeed expose it's methods and properties to xStream - the recommended approach, anyway. Many roads lead to Rome.

 

And, just a quick tip: if you want/need your file to be a class instance, then you could specify it as "userdata" - because, creating a class instance within the main loop in xStream would cause it to be redefined, over and over again... 


Tracking with Stuff. API wishlist | Soundcloud


#168 g8tr1522

g8tr1522

    Member

  • Normal Members
  • PipPip
  • 25 posts
  • Gender:Male
  • Location:Gainesville, FL

Posted 17 November 2017 - 02:15

So I'm finally trying to start adding my own code to xStream. 

 

And, just a quick tip: if you want/need your file to be a class instance, then you could specify it as "userdata" - because, creating a class instance within the main loop in xStream would cause it to be redefined, over and over again... 

 

Just to make sure, does this mean there are only two ways to add user code to xStream?

(1) by adding a property (which is my module) to the sandbox in "xStreamModel.lua" like I described earlier (which means its required in main.lua as well) 

(2) by "specifying it as userdata" as you mentioned

Also technically (3): adding a sandbox function (ie, with sandbox.str_prefix/suffix).

 

How do I do method (2)? Do you mean to just add my module like any model in the "./userdata" folder? 

 

I was able to get method (1) to work just fine. But like you said, if I instantiate an object from my own class, it'll be re-instatiated over and over as xStream traverses the pattern.

 

 

This got me thinking about how xStream compiles/executes its code. 

 

In any xStream model, when does the user code get evaluated? I'm guessing that for every line traversed, xStream evaluates the entire code. More specifically, it does load(sandbox) (or whatever) at every pattern line.

 

But what about the classes in the xModel.lua sandbox? Let's say I call 'xLine' in my code. Does xStream precompile the bytecode for xLine, and it just 'calls' the parts it needs to when xLine is called (in my code)? Or does the xLine module get recompiled every time it gets called (along with the sandboxed code)? 

I'm assuming it's the same if I add my own module using method (1).

 

What is different about adding my module as "userdata"? I'm guessing that xStream pre-compiles my module. And when I call a function from it, it'll jump to the already compiled and loaded bytecode. 

 

 

I really should get a better grasp on the compiler and enviroment before I start asking questions like this....

But I'm asking because it seems like I could use method (1) for single-use functions, and I'd stick with method (2) if I wanted to make a class system. 

 

I plan on making a big class system that'll stream-line writing xStream models (or just Renoise scripts in general). Before I start getting into it I wanna make sure I put the right code in the right place. 



#169 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6492 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 17 November 2017 - 12:27

(2) by "specifying it as userdata" as you mentioned


Ehrm, sorry, let me clarify this:
-> data is referring to data that you want to reference in your loop. You can add an initial value using the "+", or simply define them on-the-go.
-> userdata is referring to a custom path, so you specify a folder somewhere outside the tool folder. Avoids that your models get overwritten on tool updates and such.

They are not related in any way - no precompile step or anything clever like that. But originally, they were both named userdata.
You're not the first one to get confused by this

 

In any xStream model, when does the user code get evaluated? I'm guessing that for every line traversed

 

Yep. The same line can even get evaluated multiple times, if an argument that has been set to "re-buffer" has changed it's value. 

 

But what about the classes in the xModel.lua sandbox?

 

It depends. Most of xStream/xLib are based on static methods, and reside in the global variable space (in the scripting console. try entering rprint(_G) and see a complete listing of what is available. Doing the same in xStream will reveal nothing, because it's hidden/sandboxed). Static methods are usually preferred over instances, as they don't need to be instantiated - they are already there. But obviously, you will sometimes need an object that specifies it's own properties - a ClassName() will allow you create a new class instance. Luckily, garbage collection in lua is pretty efficient so it doesn't matter that much. 

 

Btw: I'm hoping to get the next version finished soon, sounds like you have some good ideas lined up  :)


Tracking with Stuff. API wishlist | Soundcloud


#170 g8tr1522

g8tr1522

    Member

  • Normal Members
  • PipPip
  • 25 posts
  • Gender:Male
  • Location:Gainesville, FL

Posted 17 November 2017 - 20:47

Ehrm, sorry, let me clarify this:
-> data is referring to data that you want to reference in your loop. You can add an initial value using the "+", or simply define them on-the-go.
-> userdata is referring to a custom path, so you specify a folder somewhere outside the tool folder. Avoids that your models get overwritten on tool updates and such.

They are not related in any way - no precompile step or anything clever like that. But originally, they were both named userdata.
You're not the first one to get confused by this

 

More confused now actually, lol  :rolleyes:

 

Okay, so userdata is just that userdata folder, where user models are kept. Nothing else, right?

What is data now? Is it self.data in xStreamModel.lua (line 85)? 

 

I guess I didn't clearly state what I'm wanting to do. I want to be able to instantiate an object from a class, to use in my xStream model. But I shouldn't do that inside of the model because of the problems mentioned before. So where do I put my class, and where should I instantiate the object? 

 

Come to think of it, it's probably fine for me to put the class in the xStreamModel.lua sandbox. It's just initializing the object is what I'm having trouble with.

 

In my fantasy world, I could do this to instantiate an object:

    S,L = xpos

    if (S==1 and L==-1) then

        obj = MyClass:new()

        obj.notes = {4,8,11}

    end

So we would never hit line -1, but we'd still enter the if block somehow...

 

 

Also, do my classes absolutely have to be luabind classes? This is what's recommended according to the classes.lua snippet. Not to mention it's what xStream classes are made with.

When I was putting classes in the xStreamModel.lua sandbox, they were working both as luabind and 'ordinary' lua classes. ('ordinary' class being where the class is the metatable for the object).


Edited by g8tr1522, 17 November 2017 - 21:04.


#171 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6492 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 17 November 2017 - 21:19

Instead of trying to explain the concept, I would suggest you take a look at some of the models that make use of "data", such as the arpeggiator or LFO?

They both take advantage of the feature to add helper methods, are being initialized with specific values, etc. 

 

Essentially, data is just there for convenience, a way to access and define numbers/tables/functions/etc, so you can keep the main loop clean and focused.

In theory, you could define your (non-luabind) classes there too. 

 

As for adding your own objects to the sandbox, I simply regard this as yet another way to achieve the same result.

The difference would of course be, that whatever you add to the sandbox would be accessible across all models. 

 

Also, do my classes absolutely have to be luabind classes? This is what's recommended according to the classes.lua snippet. Not to mention it's what xStream classes are made with.

When I was putting classes in the xStreamModel.lua sandbox, they were working both as luabind and 'ordinary' lua classes. ('ordinary' class being where the class is the metatable for the object).

 

I'm using them throughout, but you don't have to. There are tradeoffs and advantages to using either approach. 

For example, you can't use the __index meta method in luabind - which is required for some of the more advanced techniques (the sandbox is therefore also not a luabind object).

Luabind, on the other hand, offers a clean syntax and the ability to specify handy getter/setter methods.


Tracking with Stuff. API wishlist | Soundcloud


#172 g8tr1522

g8tr1522

    Member

  • Normal Members
  • PipPip
  • 25 posts
  • Gender:Male
  • Location:Gainesville, FL

Posted 17 November 2017 - 21:44

In my fantasy world, I could do this to instantiate an object:

    S,L = xpos

    if (S==1 and L==-1) then

        obj = MyClass:new()

        obj.notes = {4,8,11}

    end

So we would never hit line -1, but we'd still enter the if block somehow...

 

Praise be the lambda calculus...I'm onto something

 

I was thinking if I had some kind of latch - which would allow the if conditional to evaluate to true only once - then the if block would only execute once.

All thanks to closures, I can define a function that : returns a 0 the first time its called, but nil for any successive calls:

latch2nil = function()

  latch = function (val)
    if val then
      val = nil
      return 0
    else
      return nil
    end
  end
  

  i = 0

  return latch(i)
end
 
And then in my xStream model, I put something like this:
if (xinc==latch2nil()) then
  print("I'm only here once :(")
end
 
Now, the if block executes only once! At least, until I hit the play button in xStream again.
 
 
Gotta love how lua functions have lexical scoping. I guess I could get the same result without using closures, but this is much nicer.
 
I'm gonna keep playing around with this to see what I can get away with... 

Edited by g8tr1522, 17 November 2017 - 21:55.


#173 g8tr1522

g8tr1522

    Member

  • Normal Members
  • PipPip
  • 25 posts
  • Gender:Male
  • Location:Gainesville, FL

Posted 17 November 2017 - 21:55

Instead of trying to explain the concept, I would suggest you take a look at some of the models that make use of "data", such as the arpeggiator or LFO?

 

I'll definitely look at that, thanks!

 

But because of what I just came up with (the latching function), I'm gonna start with this instead:

My class modules are all xStreamModel sandbox properties.

Any objects I instantiate will be done using that latching function.

 

The only problem is if I want my object's notes/delays to maintain states between each xStream session. I'll figure that out later though. Gotta do what's easy first.

 

So now I'll stop bothering you. At least for a little while...  :P


Edited by g8tr1522, 17 November 2017 - 22:09.


#174 g8tr1522

g8tr1522

    Member

  • Normal Members
  • PipPip
  • 25 posts
  • Gender:Male
  • Location:Gainesville, FL

Posted 19 November 2017 - 01:31

Alright, well I lied. I'm back.
 
So, I'm trying to add my own classes via sandbox properties in xStreamModel.lua. I'm having trouble adding more than one source file.
 
I want to add a single class, eg "HelloClass.lua". Just to make sure this is a legitimate strategy, I put all the steps I used to accomplish this:

Spoiler

The code for HelloClass.lua is this:
Spoiler

Basically, it just counts the number of objects instantiated, and prints out table locations.
I used this to get an idea of how xStream would behave when creating objects from my user-defined classes. 
 
So yes, this did work! I was able to use it in an xStream model, with some success. There were issues with it, such as number of objects created. But I wanna ignore that for now.
 
 
The issue I'm having is that I can't add more source files like this.
 
I want to add another class, "HelloClass2", which is exactly the same as "HelloClass" - except there's a '2' anywhere after 'HelloClass'. I repeat all those steps above (and those related lines for HelloClass2 are un-commented). When I try to refresh the tools/open up xStream again, I get this error message:
    *** main.lua:168: attempt to index field 'preferences' (a nil value)
    *** stack traceback:
    ***   main.lua:168: in function <main.lua:166>
 
    *** Could not attach to sample - none selected  
 
I tried this again by making HelloClass a luabind class. Same results.
 
 
So yeah, is what I'm trying to do okay? I won't break xStream by doing this for a bunch of source files?

 

Or am I being way too stubborn, and I should stop begging you for help every day?  :D  But seriously, I won't take offense, haha. I've actually been looking up ViewBuilder stuff. Just to see how hard it would be to make my own version of xStream... if I can't make this work.


Edited by g8tr1522, 19 November 2017 - 01:41.


#175 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6492 posts
  • Gender:Male
  • Location:Berlin
  • Interests:wildlife + urban trekking

Posted 19 November 2017 - 11:59

I think you're experiencing some of the same issues as I've come across. Basically, that xStream is such a large tool that some internal limitations/restrictions in lua are starting to apply. 

http://forum.renoise...tream/?p=357815

 

Still not sure exactly what triggers the problem, but, hm, I do know of a workaround: removing key shortcut /midi mappings from the main.lua file. 

At least, that has worked until now and hopefully, will be enough until we can solve this issue properly (as in: compile the built-in lua component with the ability to load more resources)


Tracking with Stuff. API wishlist | Soundcloud






Also tagged with one or more of these keywords: live coding, sandbox