Using custom FX numbers+values for storing tool data. Thoughts?

Renoise Tool Development
1

I really like the idea of storing track comments, made possible by a tool from fladd. (http://www.renoise.com/tools/track-comments)

What I really want, though, is pattern comments. (EDIT: Not quite. See additional edit below)

I looked at the code for the track comments tool and saw that fladd very smartly used track names/numbers to anchor the comments. Observers on track edits are used to update the comment index (which are stored in the song comments field).

Patterns (as best I can tell) do not have an editable identifier. You cannot name a pattern. You can locate a pattern using a track name and a pattern index, but then you have to carefully watch for pattern insertions/deletions. (EDIT: Not really. I’m confusing pattern withPatternTrack.)

EDIT: Patterns can be named. Pattern names can be edit/read via Lua. But now I see what I want is PatternTrack comments.

After some experimenting I found that you can edit an fx column entry to hold almost arbitrary values. There are some range constraints, but the value does not have to have any interpretable meaning. For example, I can enter NG A1 as an fx command. It has no meaning to Renoise; Renoise (happily) just ignores it. But custom tool code can use it to associate data.

My scheme now is to anchor pattern comments by inserting a custom fx value; each would be unique to the song.

This would allow for patterns (or comment anchors) to be moved around without losing the association with the comment data.

(This also allows for per-line comments.)

My casual empirical usage suggests that such “nonsense” fx values do not interfere with the regular operation of Renoise.

One obvious downside is that there are only 8 available track fx columns; using one for comments might interfere with a later desire to have all 8 columns available for something else. I find this a negligible issue, one I think is rare to happen in actual use. And if a comment fx is in the way it can be moved to another line.

Are there other possible issues? A future version of Renoise might become strict about fx entries. I’ll worry abut that if it ever comes to pass. I’m concerned with some edge case with the current version of Renoise, something I’ve just not considered because of my own workflow.

Thoughts?

2

My spontaneous reaction is that using pattern data for storing custom data is too ugly. The benefit is that the user gets a visual cue that the patterntrack probably contains a comment. I imagine you’re gonna show this in a pop-up window opened via a shortcut? Anyway, I think it’s better to use song comments since I don’t like the obvious cluttering. I think this is the most common solution that other tools have used?

Using song comments requires some notifier handling when things are swapped/deleted/inserted. Integrity must not fail!

The last option is to wait for updates. Danoise hinted that he also thinks custom data on any Renoise object is desired. How far into the future it would be realized, I don’t know :slight_smile:

3

A simple option would be to use note column names as data containers. I don’t think there are many users that use these for something else (if at all). At least not all 12 of them, and you could occupy the first free one from the right.

These columns are per track and not per patterntrack, but you can store many characters here. Something like

<pattern 3>Try a tom fill-in here</pattern 3>

</patterntrack comments>

Still you have to use notifiers for updating any shuffled data on pattern/sequence changes (like if the user has enabled “keep sequence sorted”… and I’m pretty sure these shuffles are observable in the API).

This is probably what I’d do! It should be the least complicated solution other than using pattern data. And, given the circumstances, it’s clean enough imo.

4

My spontaneous reaction is that using pattern data for storing custom data is too ugly.

It’s four characters off to the side. No different than when storing device automation commands.

But, personal taste.

I like that there’s a visual clue that there is something associated with the track pattern.

The current Track Comments tool uses the song comments as a general data store, but extracts each track’s comment into a separate popup window when the tool is invoked.

5

A simple option would be to use note column names as data containers. I don’t think there are many users that use these for something else (if at all). At least not all 12 of them, and you could occupy the first free one from the right.

These columns are per track and not per patterntrack, but you can store many characters here. Something like

<pattern 3>Try a tom fill-in here</pattern 3>

</patterntrack comments>

Still you have to use notifiers for updating any shuffled data on pattern/sequence changes (like if the user has enabled “keep sequence sorted”… and I’m pretty sure these shuffles are observable in the API).

This is probably what I’d do! It should be the least complicated solution other than using pattern data. And, given the circumstances, it’s clean enough imo.

Using note column names would be similar to what the current Track Comments tool does, and has the same issues when trying to couple data to individual track patterns. It requires exact monitoring of track pattern changes. Too complicated.

In any event, I am specifically looking to see what issues might arise from using custom fx column entries.

6

Two potential flaws:

  1. Cutting/pasting patterntracks would delete the comments. They would be tied to the note data rather than the song position, so it’s more like “note data comments” and not “pattern track comments”.

  2. The effect numbers used to store the data might be added to future versions of Renoise :slight_smile:

7

Danoise hinted that he also thinks custom data on any Renoise object is desired

Yes, in this case it would make a lot of sense.

Neurogami has point in that there would be a lot of code involved just for maintaining the values, as patterns and tracks are moved around, duplicated etc.

If you required that all patterns and tracks had a (unique) name, then at least you could identify things more reliably. But this would basically shift responsibility to the user. It’s not “really” a solution.

So yeah, even if we could come up with a workaround - which is actually tricky in this case - it would make more sense to simply allow custom properties through the Lua API.

8

On this side note… If those pattern observables gives the same callback as track observables do ({ type, from, to }), it’s about 15 lines of code to ‘interpret’ the change correctly, and a little extra to search and update the containers for keeping it in sync. So it’s not very complicated imo, if I’m not completely mistaken. It appears a bit complicated, but I’ve kept track of tracks in such a way without much effort (syncing a table perfectly to renoise.song().tracks). The notifiers/API are very sequential and reliable, even if they’re flooded.

Mainly, I just wanted to promote the possibilities and simplicity of the API here :slight_smile:

9

Two potential flaws:

  1. Cutting/pasting patterntracks would delete the comments. They would be tied to the note data rather than the song position, so it’s more like “note data comments” and not “pattern track comments”.

  2. The effect numbers used to store the data might be added to future versions of Renoise :slight_smile:

Cutting and pasting doesn’t delete anything. This is a very nice side-effect of using the pattern track fx columns for ad-hoc tool data.

You can try this yourself (which is what I just dd to verify it): Manually enter a custom value and amount into an fx slot (e.g “NG 1C”). Then cut and paste that pattern track to another pattern track. All of the data goes with it: the notes, the volume, delay, etc data, and all the fx column data.

My current code gets the current selected track index, and the current selected pattern index, and uses that to look at lines and fx columns. It then ether reads any custom items found, or writes a new one (depending on whether the tool is ot read an existing comment or add a new one.).

The fx columns entry is just a reference value that is a key to text stored in the song comments (this is how the fladd’s Track Comments tool works, except it uses track names/indices).

These ad-hoc fx values are persistent if you cut and paste (or copy and paste) a track pattern.

My code doesn’t need to track much of anything. If a pattern track is deleted, and it held an ad-hoc fx value, the corresponding comment data would still exist in the the song comments. I’m undecided on whether this is good or bad; I like the idea of not accidentally losing data. I would most likely add an option to clean out referenced pattern track comments.

The goal is not to tie data to a fixed position in a song but to specific pattern tracks (which might move as a song is developed). My own use case is that if I move a pattern track while keeping all the same content then I’m going to want to maintain the comments for that pattern track content. If I want to relocate a comment I can cut and paste the fx value. (And I can re-use comments by entering an existing custom fx item.)

As for a next release of Renoise breaking this: The release cycle of Renoise means I will be able to use this tool for quite some time before that happens : )

10

Mainly, I just wanted to promote the possibilities and simplicity of the API here :slight_smile:

I appreciate that. But (after some experiments) It seems that using fx columns removes a lot of tracking code that was needed for the original Track Comments tool.

11

Good points. Yeah, storing it as fx seems to be a reasonable compromise.

(PS. Maybe it’s a good idea to use a constant fx character as identifier, in case some other tool wants to use non-native fx values in a similar fashion later on. Technical ideas: I’m guessing you’ll use bit/string operations to encode/decode a string to 7-bit ascii, for example. Perhaps even a hard-coded huffman table that makes alphanumerical characters lighter would be a good idea.)

12

Good points. Yeah, storing it as fx seems to be a reasonable compromise.

(PS. Maybe it’s a good idea to use a constant fx character as identifier, in case some other tool wants to use non-native fx values in a similar fashion later on. Technical ideas: I’m guessing you’ll use bit/string operations to encode/decode a string to 7-bit ascii, for example. Perhaps even a hard-coded huffman table that makes alphanumerical characters lighter would be a good idea.)

Current plan is to use “NG” as the value, leaving 256 hex values for the amount. There’ll be a tool-global counter that increments as comments are added. That puts an upper limit of 256 on the number of comments. To make life easier I do not plan on tryng to re-use values for deleted comments.

The comments themselves will be stored in the song comments using a field delimiter and the “fx” key.

(The caption on the dialog window is wrong; I’m hacking on the original Track Comments tool code to see what works.)

Individual comments would then shown/edited using a custom dialog.

13

Ahh. Good idea!

(Let me know if you’ll have to use any song-wide scanning at any point for this. I have a scheme for that, being a lot faster than the usual iterations.)

14

Ahh. Good idea!

(Let me know if you’ll have to use any song-wide scanning at any point for this. I have a scheme for that, being a lot faster than the usual iterations.)

Thanks. I probably will need this. Current plan is to not do any housekeeping unless explicitly requested, so as to preserve data. But if requested the code would need to find every custom fx entry and match it to a stored comment. Any unmatched comments would be then removed.