rnsgit: Wrapper script to help manage versioning songs with git

Renoise Tool Development Renoise XML Fileformats & Tools
1

I kept thinking abut how nice it would be to manage song versions in git. Tracking the binary xrns file had a few problems. Every commit meant a complete copy of the xrns file, even if all I changed was track headings. Most changes are likely to occur in Song.xml but you would not be able to see what they were.

The better way would be to track theexpandedxrns file. This would mean unzipping the xrns and adding each file to the git repo. Any time you changed the song you would need to unzip it to get the files.

If you changed branches or rolled back to an earlier commit you would need to rezip the files back into an xrns file.

This all seemed tedious and error prone. But,conceptually, it’s not too bad.

So I wrote a Ruby program to manage this.

Big warning: I, of course, want this to work, and protect my files, and not screw up anything, but I’m far from knowing that I’ve shaken out all the bugs and managed all the edge cases. It’s very much at the “Works for me” stage.

The code is a wrapper around command-line calls to 7z and git. You must have those two programs installed.

There is probably some stuff that assumes some things that might only be true for the specific versions of 7z and git I happen to be running.

I want this to work on Ubuntu, Windows, and OSX, but so far have only been hacking around on Ubuntu.

The current code grew out of some proof-of-concept scripts; there are no unit tests here. (I do have a test script I’ve been running for me to do some quick pass/fail executions, but that’s not part of the repo.)

Get the code here: https://github.com/Neurogami/rnsgit

The repo README has more detail on usage.

2

This is intriguing stuff, and something I’ve been thinking about as well.

Not just for keeping track of your own changes, but for being able to collaborate on a song - I’m thinking, down the line such a thing could even be semi-automatic.

Biggest challenge would be that a text diff is obviously isn’t “aware” of what the data represents.

Some frameworks for XML tries to expand on this by being able to detect when a node order has changed - something a text-based diff won’t be able to catch.

That would be a step in the right direction - then add Renoise-specific intelligence, and we would be able to pinpoint e.g. a changed order in the pattern sequence.

This would obviously be a massive amount of work, as you would need to create different ways to visualize those changes as well.

But one can dream of a way to collaborate that doesn’t require a degree in information technology :slight_smile:

3

Not just for keeping track of your own changes, but for being able to collaborate on a song - I’m thinking, down the line such a thing could even be semi-automatic.

At the moment there is nothing in place that pushes or pulls the repo. The assumption is that everything is local. But since it’s git under the hood it’s certainly doable. I need to change the code such that the default behavior for any unknown arguments is the assume they are git commands. Then you can push or pull when you like.

I’d be leery of automatically pushing and pulling; for code projects I generally track small local changes and push when I have something I’m confident about. Having not collaborated on a song using git I don’t know what the appropriate workflow might be.

Merge conflicts can be an issue. More insidious would be merges that go through but don’t make sense to Renoise, leading to an unplayable song. Too be really clever the tool might need a Song.xml validator.

Biggest challenge would be that a text diff is obviously isn’t “aware” of what the data represents.

Some frameworks for XML tries to expand on this by being able to detect when a node order has changed - something a text-based diff won’t be able to catch.
That would be a step in the right direction - then add Renoise-specific intelligence, and we would be able to pinpoint e.g. a changed order in the pattern sequence.

This would obviously be a massive amount of work, as you would need to create different ways to visualize those changes as well.
But one can dream of a way to collaborate that doesn’t require a degree in information technology :slight_smile:

If you do a git diff you can see what parts of the XML changed. There’s enough context provided to know what’s going on if you know how Song.xml works.

It’s a far cry from a nice tree representation but still very usable. I tend not to use GUI tools with git (except when managing merge connflicts, then I’m running kdiff3) but perhaps one or another git tool makes viewing diffs a little nicer than scrolling in a terminal window.

edit: This brings up kdiff3 for me (since that’s what I’ve configured)

git difftool master test-branch

Makes it easier to see what’s changed, but it’s not an XML diff tool as such.

edit2: Just noticed I have now made 11111111 posts. In binary.

◕ ◡ ◕

4

I also posted over on the Reddit forum, and there was a suggestion of having something like this built into Renoise (or made available as a tool),

http://www.reddit.com/r/renoise/comments/36ybyb/rnsgit_commandline_tool_to_help_version_renoise/

Turns out there exists libgit2:http://libgit2.github.com/

And luagit:https://github.com/libgit2/luagit2

https://git-scm.com/book/en/v2/Embedding-Git-in-your-Applications-Libgit2

Perhaps someone with better skills in C+Lua would find this an interesting challenge. :slight_smile:

5

There’s a notable bug in the program, on Windows. 7z barfs on certain file names. I need to correct for spaces in file paths and possible do something about long file names on Windows.

6

This seems rad. I would love to take a crack at either messing with what you have or working on a new approach from the ground up.

Could we build this as a tool in Renoise? That would be nuts awesome.

7

I’ve discovered a horrible bug. I think I’ve fixed it, but the code has not yet been pushed.

Basically, rnsgit was extracting the current (and presumably altered) xrns file contents over the existing file contents being tracked in git.

In retrospect this seems obviously bad, but that’s how it goes.

The upshot was that SampleData would end up with a weird set of instrument sub-folders:

Instrument19 (_Percussion_130_B2)

Instrument19 (piano_1)

Instrument20 (guitar_005_01)

Instrument20 (piano_1)

… if new instruments were added in the middle of the existing instrument set.

I will be testing my code changes and then pushing them to github.

Update: After catching Yet Another Bug I think the current release does what it is supposed to do.