Collaboration/syncronisation

Renoise Tool Development Renoise XML Fileformats & Tools
14

Ok, so far there’s 2 primary ways to design multi-author online xrns sync.

  1. Many authors can work on the file at the one time… If me and you are working on Pattern 5, and you decide to modify the snare and insert a track, and modify the EQ. And I decide to add 3 tracks, and I add a reverb DSP to track 6 - and we upload at the same time, the system tried to ammend both of our ideas into the one track. The system would likely turn pattern 5 into 2 separate patterns - label one as mine, and the other as yours. It’s chaos in the making.

  2. Only one author can work on a track at once … which means those issues in method 1 will not exist. Only me or you can work on the file (or anyone else for that matter) - If i’ve been charged with working on the file, then nobody can update the file but me.

Idea 1 seemed cool, but after some thought, I decided it’s too ambious and a pain in the arse. The reason why I wanted to pull apart the XML file was so I could deal with the complicated issues of idea 1. You can’t simply diff it, as the XML requires interpreting so it knows that pattern 5 needs duplicating. We’d need to develop a heavily propietry ‘diff’…And even if we developed an amazing algorithm, it was still going to be chaos. Life is too short.

Idea 2 seems a lot more realistic, and involves NO xml parsing whatsoever. Because XML files (compressed) are small, there’s no need to deal with the differences in the XML file, therefore no need to analyse the XML file at all. It’s just a file, like a flac file. You treat the xrns as a directory structure and send/receive the differences. If the XML file has changed, send the entire XML file.

15

Here are some ideas to add some value to this simpler #2 idea.

  1. File history is kept, file can be reverted and restored based on a timestamp query. I want the song from 48 hours ago? Done.

  2. Deadlines. The user gets an email or and SMS message saying their time is up and if they don’t submit their changes within 30 minutes, another user (in a queue?) will have the next exclusive access window to the file. If I’m collaborating, I don’t want my partner to hog all the time on the track. Time windows can be enforced?

  3. Statistics can be generated from the revision history, make cool collaboration chats that look like MS Project time lines. Who worked on what, when? A visual representation. Pie charts? Changes over time? Or at least a (SOAP/XML?) interface to the data so third party users can query the Renoise Collaboration server to gather statistics?

… ?

16

File History. Hmm. I was thinking that. Now the only problem with this is database overhead. I was thinking more of storing the file history on the client side. So … “mytune_081507_1546.xrns”, “mytune_091507_1115.xrns” … obviously the sync app will look at “mytune.xrns” as the tune to sync…

Doing it the database has more advantages though, although if you’re working on a 50mb file - ohh man… 10 versions of the 50mb file is going to be paaainful on mysql. 50mb is painful enough.

The deadline idea! Great idea!!! Time windows are good, I can’t stand waiting around for days and days. I want to track, and I want to now. :P

The visual stuff sounds like a good “rainy day” idea.

17

I’ve been having some of the same thoughts, so I’ll chip in with the approach I had in mind.

I would write up a “session server”, an application that is responsible for maintaining collaboration session state - keeping track of who’s participating, the changes they contributed to patterns, instruments, samples, etc.

I believe the biggest problem here is how to keep things perfectly synchronized. I think we need to think in terms of “eventual consistency” - a data replication principle that is frequently used in clustered databases.

In a sense, you can think of the “session server” as the “database” for the song - and each collaborating client needs to be able to replicate changes submitted to the “session server”, resulting in eventual consistency; every copy of the song being up to date.

The internet being pretty fast these days, we’re talking about synchronization problems that are probably going to be pretty rare - of course, these must be handled, or you would end up with differing versions of the same song on different clients, but with a fast internet connection (or especially on a LAN connection) the number of conflicts encountered should be pretty low.

Here’s an article explaining various patterns for implementing eventual consistency:

I’m not sure which variation would be the most appropriate.

Under any circumstances, I think the pattern-consistency needs to be implemented at the track-level - for example, for every change you submit, the session server would assign it a running serial number unique to that particular track in that particular pattern, and check for a conflict; e.g. check if another user submitted a change to that track/pattern that has not yet been streamed back to the other clients - if so, you have a conflict.

Now, if every client assigns a serial number to every change as it’s made locally, before submitting it to the session server, a conflict can be resolved by telling the client in violation that their change was rejected - the client can then simply undo from it’s current local change number back to the change number in violation; the last change (or maybe a couple of changes) would “roll back” on that client. And with the “rejection” response from the session server, that client would receive the conflicting changes, and apply them immidiately after undoing the rejected changes.

For changes to instruments, samples, automation, etc. the same mechanics can be applied - locking and conflict resolution would happen at the instrument/sample/automation-graph level.

Anyway, those are my thoughts for the time being.

Network tracking was a dream of mine since 15+ years, so I was quite excited to see Renoise adding a scripting engine with a network API. I think it’s possible! And the internet is fast enough to make this truly useful now.

I regret that I’m no longer making music. Although the prospect of being able to collaborate with friends abroad is a tempting prospect that makes me think about picking up tracking again :wink:

18

It’s all about keeping things up to date. What about storing only the diffs instead of storing the whole song zip ? Tip

19

I’d like to chip in an idea, because this seems all too close to a tool I use every day. I’m an architect by profession, and the application camp I’m in is the Graphisoft one; namely ArchiCAD. While the program can on occasion be a pain in the side, it has one really handy feature and that is the current, new version of TeamWork (add copyright sign)(add registered trademark sign)(add corporate logo)(no animals were harmed). It basically means that several people can work on the same project, which is held as a database(of some sort) on a dedicated server. Like this thing you’re discussing. And the way this is pulled off, without the chaos implied in the scenario, is by reservations policy. Everyone can work on the same project, but only if they have “reserved” that specific thing they need to work on. After the thing one reserved has been completed, the reservation on the thing can be released, submitting the changes made. Changes made by the person working on the reserved thing can be submitted at any time, in which case only the ‘delta’ (or diff?) will be sent and handled by the server. Other changes, made by other people working on the project, can be “fetched” likewise. The main catch on this approach is that the reservations on things must be kept in sync. And there’s a local copy of the whole project.
While I have zero knowledge on actual networking mechanics, I’d imagine this COULD be a viable solution for working on renoise tracks too. It could mean for example that one could reserve a specific pattern to work on. Or reserve the comments-section. Or a specific instrument. One feature of said TeamWork©®(x) is that users can freely add stuff, while modifying and removing things must be via reservations.
Well. Just wanted to note this ‘partial reservations stuff’ as an application on the very subject being discussed. Carry on.

20

i think we have the same idea in mind but let me clarify what i think:
most of the suggestions have something to do with the project file being treated as a directory, unpacked, the changes diffed, etc. the CVS approach is ok, if it works for software why shouldn’t it work for other things.

since the scripting engine allows for networking and offers most of the editing functionalities, a clean approach that would even allow for realtime collaboration would be:

  1. person A connects to a server
  2. Renoise monitors all of person A’s edits (events) (i don’t think this is implemented in the API but it shouldn’t be that difficult)
  3. the edits are translated either to XML or a smaller non-readable format (see marshalling) and sent to the server. if someone adds a sample it is uploaded to the server and kept there until all people participating in the collab have downloaded it, then it is deleted. deleting a sample in your project just sends a message to all clients to remove (maybe not physically) the sample from the project.
    two important things: a history is made and uploaded to the server for all clients that were not connected while the changes were made. everything at this point is being compressed!
  4. person B receives the changes, the samples are downloaded, and they are applied by using the API again. this could technically happen (almost) in realtime or by a notification that a new version is available. this way a lot of people could participate and even watch what’s happening in realtime.

throw in the IRC plugin and you’re good to go. ^^

that’s how i would expect an idea like that to work and it does go in the direction of a SOAP approach but i would make it more dynamic … unless you really want to keep it basic.

21

I think this is unnecessary and adds complexity - simply having the most recent change override another change is probably much simpler and perfectly acceptable… a song is a big playing field - the odds of two players colliding is really rather minimal.

Have you ever worked two people on a spreadsheet in Google Docs, at the same time? It’s a fascinating experience, you should try it :slight_smile: … The tracker is basically a spreadsheet, so you can get a feel for how this could look and work with a spreadsheet in Google Docs - you can see the other “player’s” cursors (in different colors), you can see their edits in real-time, you can see them enter and leave the session, and you can chat with them. If you could see a ghosted version of the other collaborators’ mouse pointers too, that would be awesome. If the cursor left a subtle trail when changes were made (that fades or disappears after a few seconds), that would take some of the “spookiness” out, so you don’t suddenly realize that someone made a change where you were looking.

With the low latency on modern internet connections, you don’t need to worry much about the lag - participants will see the other editors’ cursors and pointers, and they will naturally get out of the way. I’ve been working with 4 different people on the same spreadsheet (relatively small, perhaps the eqivalent of 3-4 patterns in size), and we never once collided or overwrote each other’s changes. I don’t think manual “reservations” are necessary or pratical.

22

I seem to have a hard time not thinking about this - here are some thoughts I had last night…

Initially, my thinking was that one instance of Renoise would be the “master” and others would connect to it, and they would try to keep things in sync between them.

There are two problems with that approach, that stem from the same issue:

Add a simple “version control” system - meaning, whenever the song is “committed”, we keep a numbered copy of the previous version of the song. Pretty simple, and should be optional of course, in case you’re working with a huge song file.

That’s all from me at the moment :slight_smile:

23

Hi

Found this thread while looking for something similar.

  1. What’s the status of renoise not being able saving to/writing from dirs (as opposed to .xrns-files)?

  2. I’m looking at git with hooks and smudge/clean. It should be able to work with xrns files and zip/unzip behind-the-scenes, I just can’t get my head around it. Anyone tried this as well?

24

The idea of using a version control system had crossed my mind, but I don’t think it’s going to work well in practice - version control systems are designed for working with source-code, and they are not designed for real-time applications. They are also not guaranteed to automatically resolve many conflicting changes, as would occur in a system trying to use a version-control service in real-time.

It might be feasible to create an integration for a version control system for version control, however - which might be a really cool thing to have, too. So that saving your song would actually create a revision, and so that you could undo changes made by a particular revision. I’m sure there are references between objects stored in a Renoise XML file though - for example, instruments reference samples, so even if the version control system could successfully track changes and patch/remove change sets, you might end up with corrupted files that won’t actually load.

I’m not going to say it wouldn’t be cool if you could get that working - but you should not expect it to work in real-time…

25

It looks like the best algorithm around for doing this kind of thing, is Operational Transformation:

This algorithm eliminates favoritism - everyone is editing the shared data on equal terms.

Here’s an implementation with a server written in CoffeeScript and running under Node on a webserver, with a client-library (also written in CS) running under JavaScript engines in conventional web browsers:

http://sharejs.org/

Pull up two browser windows side by side and edit the text in either window - ask a friend to pull it up and do the same. Pretty amazing. Scroll down that page for more links to far more impressive examples.

It supports text and JSON, so both one-dimensional and two-dimensional data - one dimensional operations would be required for samples, while two dimensional operations could be used for patterns.

The JS client library is only 4KB of JavaScript, so it must be relatively simple. Or in other words, it’s not unthinkable that you could write a client in Lua and actually run it against a ShareJS server. This is all open source - so you could package Node and ShareJS and create a self-contained server distribution.

So the hard part here would be implementing the actual operations, or more accurately, the operational transformations on those - I guess in some sense the operations are already implemented by Renoise.

I’m not going to say this would be easy, but it’s feasible :slight_smile:

26

By the way, I still think my original idea would work, and probably much simpler to implement - but I should highlight one weakness of that approach: it interferes with undo/redo. That is, if you undo an operation, you’re not undoing an operation done by you, necessarily, just the last operation performed by any of the participants, which is probably not the desired effect.

For that matter, if an OT approach was used, undo/redo would be considered operations in their own right - that is, you would have to implement operational transformations for the undo and redo operations. I could be wrong, but I’m not sure the scripting API is up to the job - this may require tighter, more low-level integration with the Renoise editor than is possible with the scripting API. Or not - I don’t know it that well, and definitely have not been keeping up with new developments in the past 6 months :wink:

If anybody has the time (and gusto!) to attempt something like this, here’s a very good explanation of OT:

27

Just a quick follow-up with a couple of observations I’ve gathered from learning some new things since posting in this thread.

  1. If we’re talking real-time collaboration, Operational Transformation, in it’s simplest form, would only work if every possible operation in Renoise can be expressed as a “flat” operation - which may be possible, because almost every collection (instruments, samples, patterns, tracks) in Renoise (as far as I understand) is “addressable” using a flat index. Operational Transformation on anything with structure (such as nodes in an HTML, XML or JSON document) are not simple - they are extremely complicated, and most open source OT libraries do not support it.

  2. If we’re talking on-demand synchronization/merge, a straight diff (such as provided by common diff tools and version control systems) will not work, again, because an XML file, while stored as a flat text-file, does contain structured data, which is much (MUCH) harder to synchronize - which is why you will often experience merge failures in e.g. HTML, XML or JSON files with version control tools. There are specialized diff/merge tools out there designed specifically for XML (such as this one for PHP) but even these can’t always successfully/correctly merge changes to structured data.

To give an example of the “unsolvable” problem, consider an operation where an item is renamed or changes position, versus an operation where an item is deleted and another item is added - without knowledge of the operation that happened, with the end result of both operations looking the same, one can only guess at which operation occurred. Operational Transformation addresses this by working with operations rather than the result of those operations - an XML file is only going to reflect the result, not the actual operation, so an approach using the XML file is always going to be brittle or limited in some way.

The possible exception in the case of Renoise, is a custom XML diff/merge tool specifically designed for Renoise XML files - since all collections (AFAIK) in Renoise are “addressable” (as discussed above) it should be possible to treat the data in the XML structures as flat data rather than as structures, and thereby implement working diff/merge operations, though still with the limitations covered above, as far as being unable to know which operations occurred, and having to make assumptions or guesses, which could be difficult.

In my opinion, real-time synchronization may actually be more feasible to implement than on-demand diff/merge.

But as I see, the Renoise scripting API, as it is, is currently not suitable for implementing this… in order to implement real-time synchronization, the command pattern (which Renoise uses internally to implement e.g. undo/redo) needs to be fully exposed via the scripting API. You need to be able to construct, serialize/unserialize, and execute commands - and command events need to be cancelable. Until the scripting API exposes these features, you can’t implement real-time synchronization.

28

WoW , I didn’t have time to read through all the topic, but does this mean that then you could have collaboration-gig with your fellow Renoisers (almost)realtime in different countries?

29

Maybe out of the box thinking is needed, have a camera pointed at the screen or some kind of screen capturing going on and have optical recognition algorithms translate what’s happening realtime, send converted OSC instructions to a arduino controlled robot on the other side, auto-mirroring, tracking the results.

NES-ROB.jpg2640×3340

but on the real to reals, some kind of collaboration option would be nice, take turns on creating a beat without having to be in the same room.

30

That’s the dream - and with modern internet connections, it would be very nearly real-time.

31

Reading through this post a couple years later - it popped into my head again after tinkering with deepstream a couple of weeks ago.

This thing is basically the back-end we need for real-time collaboration - no need to develop a custom Renoise server or anything like that.

It uses a “last in wins” strategy, which should be fine with today’s internet speeds and the likelihood of actually making conflicting changes.

It’s open-source so anyone could host their own.

It was extremely easy to implement in JavaScript, since they have a JS client library ready to use.

Under Lua, hmm, not so much? Lue is a pretty exotic language - it’s not likely we’ll have a client for that anytime soon. It’s too bad they selected Lua as the scripting back-end. JS would have been much more accessible to a larger audience, and would have made something like this readily available to implement.

Well, either way, I poked through the API reference, and it looks like there would still be some work to do before a script could provide this integration.

On the other hand, maybe with this as a back-end, the Renoise team itself might be interested in making this happen? Not that there’s a client-library for C yet either, so I guess, either way, even though the back-end is there and ready to use, implementing a client-library might be a pretty substantial first hurdle…

32

A lot of work, but totally possible as xrns is an open format :slight_smile:

Also, neurogami has made a proof of concept of sorts, check out this thread:

https://forum.renoise.com/t/rnsgit-wrapper-script-to-help-manage-versioning-songs-with-git/43667

33

Well, the file-format is documented, sure - for a diff/patch-facility and git-support, that’s nice, but doesn’t really help with real-time collaboration. For that, we need the ability to basically subscribe and block all events, then apply them in the order the server echoes them back.

Depending on how central the event-bus in the Renoise architecture is, internally, that could be either really easy, or really difficult - if basically every command/change goes through a unified event-bus, it should actually be quite easy.