Merge Songs

Renoise Tool Development Renoise XML Fileformats & Tools
#150

Thanks a lot for the suggestion. I tried putting this line in top of both xrns_functions.php and xrns_merge.php, but whatever I set the limit to (tried large and small numbers) it doesn’t matter. It still returns with the message:

Warning: SimpleXMLElement::asXML<>: Memory allocation failed : growing buffer in C\...xrns_functions.php on line 182

Observering the php.exe working in the Process Manager, I can see that the error message sets in when php.exe is using approx. 1 GB ram (it won’t allocate more than that though there’s approx. 1.5 GB unused ram).
So, I would guess there’s another memory limit at work here but haven’t been able to find anything about it on the internet.

#151

Can you share the XRNS files that cause the error?

#152

I would be happy to share the files with (just) you, Conner.

But it’s a lot of data: They are 130 MB and 170 MB.
If you still wan’t to test them yourself, I can send them via Skype or what you would prefer.

#153

Great. Can you share a folder on Dropbox with me?

dac514 < yacht > hotmail.com

PS: This is not an email I read, it’s more of a spam catcher that I check once a month, linked to these kinds of services. If you need to email me, click my sig then go to my CV for my gmail.

#154

I’ve shared a Dropbox folder. Happy testing!

Edit: The problem seems to arise whenever the songs a loaded with instrument/sample data beyond a certain limit. E.g. it seems I can create the problem with any songs by just filling samples in them.

#155

I did some testing on my OS X machine. (It doesn’t use the XRNS-SF front-end, just uses whatever PHP I have installed at the command prompt.)

I added this to the top of xrns_merge.php, near line 18:

  
ini_set('memory_limit', '-1');

This basically says “no limit.”

The merge worked fine for me, so far.

I will do some testing on Windows in the next hour and reply again.

#156

I tried on a Windows (32bit) Netbook with 1 gig of RAM.

i can confirm the problem. I too get a “Memory allocation failed : growing buffer” error.

I tried:

  • Upgrading to PHP 5.4.3
  • Downgrading to PHP 5.3.13
  • Freeing up memory manually (see changes)

Nothing worked.

My guess at this point is that 1 GB of ram is just not enough. My Mac (64bit) has 8 GB of RAM, for example. The merge works fine.

The next test would be to try on a 64bit Windows machine with more memory. You’d have to share your files with Beatslaughter if he’s up for it? Or another tester? (Note to tester: Update from SVN first, please. You can do this by double clicking “get_svn_dev_version_scripts.cmd” in the root dir of XRNS-SF to get my changes.)

Sorry, there isn’t much I can do here. Technically speaking, I don’t want to change from SimpleXML to a leaner library. SimpleXML is simply a memory pig AFAIK.

#157

Yes, i can do that. Running Windows 7 64bit here with 8GB RAM and also have Dropbox. If Drop Shadow is ok with that he can send me an invite to the shared folder to “info <…> beatslaughter.de”.

#158

I have 3 GB ram here.
Thanks for looking into this issue. I’ve shared the folder with Beatslaughter too.

#159

With default settings i was able to replicate the running out of memory issue and adding that memory size line to the PHP script didn’t help either. I have found a solution it seems by making the PHP executable large address aware. Normally this is a compiler setting, but with this small tool you can patch the file yourself.

  1. Download the small tool from the first post here: http://www.techpowerup.com/forums/showthread.php?t=112556
  2. Pick the php.exe file in the php subfolder of xrns-sf as target.
  3. Tick the checkbox which enables the flag and then press the save button.
  4. Try the merge again and it should work.

If you’re too lazy for all that, grab the appended php.exe and change the file with the one included.

#160

I’ve applied the patch and updated the xrns_functions.php and xrns_functions.php to Conner’s latest versions.

I tried it in three different scenarios.

1. WinXP 32bit 3GB ram.
Same issue.

2. WinXP 32bit 3GB ram - With 3GB switch enabled.
It was still not possible but I could see in the Task Manager that the php.exe now topped at around 1.5GB when processing the files (before the patch was applied it was 1GB). So it seemed it got further in the process. And it gave a different error report (still mentioning memory issues though).
Still, it stopped way before all available memory were used.

3. Win7 64bit 2GB ram.
Success! I also tried with some even larger files. Still success.

So, for some reason WinXP 32bit 3GB ram can’t manage the job while Win7 64bit 2GB ram can.

In any case: Thanks for the effort. Using my laptop with Win7, I am now a happy merger.

#161

I know this great tool isn’t exactly maintained but anyways:

I’ve got this issue which is perhaps best explained by an example:
Let’s say I want to merge song 1 and 2 (in that order).
Song 1 contain instrument numbers 00-09. So does song 2 (after merging, song 2’s instrument will be reordered to 0A-13.

Song 2 consists of one track with 3 columns. In the third of these columns there’s a note playing instrument 01 (0B after merge).
Now, in the first two columns of that track there will be inserted two 0A’s in the instrument column.

So…

--- .. --- .. C-4 01

will after the merge operation become

--- 0A --- 0A C-4 0B

while it should be

--- .. --- .. C-4 0B

With complex songs, this means there will be lots of changes to the way the song sounds…

If there’s a quick fix to this, I would be grateful if anyone would make it.

#162

Can you post two XRNS that I can use to duplicate the problem?

Thanks.

#163

Two really simple songs here…

Thanks for looking into it.

#164

Thanks for that. XRNS files to duplicate bugs are really really helpful.

The problem is as follows:

Old XSD (every version before 2.8)

<element name="Instrument" type="xs:string" minoccurs="0"></element>

Newest XSD (RenoiseSong37.xsd)

<element name="Instrument" type="xs:string" default=".." minoccurs="0"></element>

The old value was always “numeric” whereas the new value can be both “numeric” and “…” - I’m not sure if this is a bug/mistake in Renoise? Anyways…

The fix is to change line ~279 from:

if ($z->Instrument) {

To:

if ($z->Instrument && '..' != $z->Instrument) {

See this commit for more details. As I am shutting down XRNS-PHP I’m not sure how to get these changes to you. The best way is to change the code yourself until something is worked out. Or get comfortable with the command line and clone my new git repo, which I will be updating from now on.

Hope this helps.

Cheers.

#165

Great. I changed the code and it works.

However, I’ve encountered a new issue.

When the second of the merged songs has a track in which a VSTi is AutoAssigned to a track, this Autoassigning will be changed, so that it will now be assigned to the corresponding track (meaning same track number) in the first of the songs that were merged.

See test examples and notice that after merging the sound output is now much louder because the original track dsp in song 2x is now no longer applied to the VSTi.

For test purposes, I’ve used the free VSTi dmihammer (also attached).

Hope this will help you in your “git repo” work.

And if you got a quick fix for xrns-sf too, I’m once again very grateful…

#166

Thanks Drop Shadow, xrns examples are awesome, you are my favourite user of the tool.

I will check this out and reply here soon.

#167

Ok I found the problem.

The issue is that I cut and paste this code snippet verbatim a long time ago, but it never worked. Nobody has reported it as clearly as you did until now.

The fix is this diff. Or replace the old routing code with:

  
// AssignedTrack routings  
foreach ($sx2->Instruments->Instrument as $x) {  
 if ($x->PluginProperties->OutputRoutings->OutputRouting) {  
 foreach ($x->PluginProperties->OutputRoutings->OutputRouting as $y) {  
 if ($y->AssignedTrack != -1) {  
 $y->AssignedTrack += $offset;  
 }  
 }  
 }  
 if ($x->MidiInputProperties->AssignedTrack != -1) {  
 $x->MidiInputProperties->AssignedTrack += $offset;  
 }  
}

The code is on GitHub. Can you work with this?

#168

Once again good fix. I changed the code in Wordpad. It works great.

Re. Github: I’m really a simple user and know nothing about coding. So I look for the executable file and when I can’t find it I don’t know what to do. I should somehow execute the xrns_merge.php inside a GitHub program … ?

Btw: found another bug.
When Song 1 doesn’t contain any samples (it’s either empty or all instruments are VSTi’s) and Song 2 contain one or more sample based instruments, merging will fail with this kind of error:

---------------------------------------  
Warning: mkdir(): No such file or directory in C:\...\xrns-sf-1.09\scripts\xrns_functions.php on line 88  
  
Warning: copy(C:\Users\.../xrns_merge_55e65da6fd51f4be6a897391  
b683b171_Track01//SampleData/Instrument10 (Hat-001)/Sample00 (Hat-001).flac): fa  
iled to open stream: No such file or directory in C:\...\xrns-sf-1.09\scripts\xrns_functions.php on line 98  
Error: File 'C:\Users\.../xrns_merge_af9b6dfbace3518aaf1cbf96b  
42e8242_Track02//SampleData/Instrument00 (Hat-001)/Sample00 (Hat-001).flac' coul  
d not be copied.

Try merging the two attached songs to produce the error.

Select 2y as the first song to be merged and it’ll work!

#169

If GitHub is unfamiliar to you, Wordpad is fine.

Again thanks for the bug report. Amazing as usual. The fix has been committed here.

Basically look for:

  
// SampleData directory  
if (is_dir($unzip2 . '/SampleData/')) {  
 foreach(new DirectoryIterator($unzip2 . '/SampleData/') as $file) {

And change to:

  
// SampleData directory  
if (is_dir($unzip2 . '/SampleData/')) {  
  
 if (!is_dir($unzip1 . '/SampleData/')) {  
 $ret = mkdir($unzip1 . '/SampleData/');  
 }  
  
 foreach(new DirectoryIterator($unzip2 . '/SampleData/') as $file) {

Hope this helps.

1 Like