Out of Memory Problems - 3GB XP & Large Samples

Interesting discussion.
I have a related question. I have 64bit Vista and 8GB of RAM. How much memory can Renoise use on my system? 3GB? Is there any way to workaround this? I am planning to use 32-bit VST instruments.
Is it possible for Renoise to do some kind of a fork() and launch each VST instrument as a separate process? That would probably mean wasting some memory and maybe some performance impact on the communication between these processes, but could it potentially solve the problem under 64-bit OS? (I have no problem adding even more memory and buying faster CPU in that case).

I see more and more VST instruments being 64-bits and having some outrageous memory requirements (just look at some latest VST instruments from Spectrasonics - 4GB recommended). Can Renoise keep up with this?

first of all, take in count that we are aware of the increasing demand for a Renoise 64bit version.

I think that the maximum quantity of memory available in Renoise under a 64bit OS is 2GB, as it’s most likely that the /3GB option is not available there.

about the separate process idea, VST’s already are separate processes loaded as dynamic libraries, so there is nothing more we can do.

Its 3GB when running Renoise version on a 64bit windows without having to set any option…

I think Cadence means something else: By creating a new process for each VST, you do not have to share the memory with Renoise and all the other plugs. So Renoise and each single plugin would have up to 3GB of memory - each. This also has the advantage that you can use 32bit plugins in 64bit hosts and vice versa.

But launching plugins in a separate process creates a LOT of overhead as well, because you can no longer directly call/communicate with the plugins.

I think reaper got this kind of bridging for plugins in one of its last updates. Probably someone here has tested this and could give us a hint how well this works, how big this overhead is?

First off, It-Alien, it is reassuring to hear that Renoise team is aware of the 64-bit issue. I don’t expect immediate solution to this, but it’s good to know it is somewhere on your TODO list.

This is exactly what I was thinking about. Although, I didn’t know it can also potentially resolve problem with 64-bit plugins.

It would be great to know how big is that overhead. To mitigate the overhead, in ideal world, it could work like this:

  1. Keep loading VST instruments the normal way.
  2. Once we get the “out of memory” signal from the OS, Renoise will try to load VST instrument as a separate process. If that doesn’t work, then it means we run out of system memory so the “out of memory” message is diplsayed to the user.
  3. if the VST instrument is 64-bit, Renoise will load it as a separate process, regardless of the current memory usage.

This way the performance impact would be only for those VST instruments that normally wouldn’t be handled by Renoise at all.

No matter which way you guys will decide to go with this, one thing is clear: it is not trivial. It will probably take months before we will see Renoise that can operate above the 3GB barrier. Therefore, in the meantime, I’m wondering if there is any workaround whatsoever. It can be dirty - as long as it works :)
One thing that comes to mind is to use two Renoise instances at the same time. When you start two Renoise programs, each one can actually operate at the same time (each can play different song for example). If there was a way to somehow synchronize them (ReWire?) that would solve my problem for now (especially that I have two LCD screens so I can work together with two Renoise’s easily). I know it seems ugly (and many features wouldn’t work), but I’m just trying to think outside of the box here. It’s really annoying that I have 8GB RAM in my system, but can use only 3GB…
Any ideas for a workaround?

Theres a plugin which can “bridge” other plugins which run in their own process:
http://jstuff.wordpress.com/jbridge/

When using it for 32bit plugins in a 32bit plugin host you could in theory use 3GB for every bridged plugin…

@taktik: this should resolve the problem for now. Thanks.

i’m back 8) i have finished my project, most of the songs excided the amount of memory with 1.4 or 1.5 gigs of ram. (pls read bottom for more)

the /3gb switch in win.ini was the solution for me, but you should take care not to run any d3d programms, the system (with my ati card) becames very unstable and freezes after some minutes with d3d usage (wxp)
(tested with 3 different catalyst drivers, it only happens with the /3gb switch)

i also have tested

and it does the job very well, but you have to buy it. but it solves the problem too! but i think renoise should handle the VSTi’s as single threads byself.

after very intensive research in the last month’s i can say:
wxp only can adress 1.34 gigbytes of ram / thread. not 2 gigs which anyone (including microsoft) claims. (!)
with /3gb switch it raises up to 2.5 gigs of ram.
and one of the major problems is: if the maximum amount of ramusage in renoise is reached (usally at 1.34 gbyte) a warning
message in renoise is popping out, that says something ybout low memory or not more memory free, and after that you can’t save your song nor delete a sample (even with undo off) to free up some memory.
you only can kill renoise in the taskswitch and load the last state of your song.
i called this popup “screen of death” . got this several times. (without the /3gb switch)

From my side I will just add that on Vista x64 I was able to successfully edit my song even when it reached 2.4GB of RAM. Haven’t tried more yet, but I will let you know if I ever hit any limits (3GB?) there.

I’m reopening this, because of one question I want to ask: what’s the problem with building x64 version of Renoise? It’s just a matter of compiler, right? I don’t need anything optimized for x64. I just want it to work in 64bits so that I can use 64bit VST instruments (I don’t need 32bits at all). Is it possible to build such a version? Even just an unofficial “testing” build would do ;)

It is not only compiler, all the internal sample processing routines and the audio engine has to be changed to also do 64-bit audio processing i guess.
Compiling a 32-application with a 64-bit compiler parameter only resolves the part where Renoise can run natively in a 64-bit environment but it will likely crash when you try to use a 64-bit plugin as soon as the plugin has to feed audio to Renoise.

Not sure if it’s the same in the “workstation” variant of the OS or if it’s the same as the server OS, problably the same.

The /3GB switch you should really be careful with and check that the applications you’re running is programmed for it. It’s the same thing with the /PAE switch that can wreck havoc depending on the applications running.
In a 32bit windows installation if you have 4GB of RAM it will allocate 2GB to application processes and 2GB to kernel mode processes. So really the maximum in one single instance of an application is 2GB.
Unless you use the /3GB switch that will change the allocation to 3GB for applications and 1GB for kernel processes.

If however you run a 64bit OS you should be able to allocate the full 4GB as that is the maximum addressable size in 32bit.
This of course require that you have more than 4GB of memory available for the OS and what not.

@haplo

pls read my post above, what you write is claimed theory by microsoft, it does NOT works.
i tried some programms (not only renoise) and no one reaches 2gb / instance. (not even 1.6gb!)
please try it byself :)
i found your text many times trough reading the internet, but that its simply false. (on my wxp / pro, sp2 and sp3 tested) .
we all have feeded google before readen trough specifications before we are confused and posted the memory problem here.

this is not working for renoise, if i understand vV right, please read vV’s last comment. you would need 64bit vsti’s, and that would resulting renoise to crash (?)

This is where it stops for me.

6GB of RAM
Vista x64

I loaded about 30+ 95MB samples. So no VSTi in this case.

Screenshot

I don’t think we need 64-bit audio processing. I think the only two reasons for having 64bit Renoise would be to be able to allocate more than 4GB of RAM, and be able to use 64bit VST instruments.
You guys can probably later add some 64bit optimizations to your code if you find any neat ideas how to speed up things on x64, but that would more of a bonus item. Not the main goal.

Haplo thanks for trying that. It is in line with my results on Vista x64. Renoise can allocate almost 4GB of RAM. I think it is huge amount if you only work with samples :) But if you have some more complicated VST instruments, you can actually hit this barrier quite quickly. That’s why many of those new VST instruments are 64bit now. So far they still have 32bit versions included too so we can use them with Renoise, but I would really love to see some 64bit support in Renoise in 2010. Maybe in version 2.7.0? :) The best would be 64bit version of Renoise with a support of 32bit legacy VST instruments :dribble:
I think 2.5.0 gave us so many new features to work with, that the next version should probably focus more on under-the-hood items to keep Renoise compatible with changing environment.

What do you think that 64-bit plugins would supply? What 64-bit plugins submit to the host is beyond Renoise its controls, or they simply just don’t make themselves available if the hosts asks them to submit 32-bit audio back.
You will only have more RAM available, but you probably still won’t be able to use 64-bit plugins if they are not allowed to submit back 64-bit audio if they really require that.
I know Taktik can shed a better light on this regarding how this stuff works from his analysis.

i don’t understand why a single plugin really need more than 1.3 gigs of ram. all “big” vsti’s with a large amount of samplelibrarys use streaming from hdd.

important would be that every vsti got his own thread. for example percussion adventure uses 150mb per instance, kontakt2 110mb, edirol orchestra 110. currently all vsti’s uses the same ram-thread, there is no much space together with some long wav’s in renoise under a 32bit system with this 1.36 gigbyte limitation of windows / thread.

also maybe renoise would be sturdier if a vsti crash’s?

That is already the case in many situations. Threading can also be done with 32-bit CPU’s or 64-bit cpu’s in compatability mode, no problems.
Memory issues is something that most likely will be come a thing of the past too if Renoise indeed supports 64-bit.
But then one should make it more common to have 8 or 16GB of ram to make this really worthwhile.

Ok, I’m definitely not an expert in this field. And I think I may be confusing things like 64bit audio (is there even something like that?) and 64bit addressing space of x64 architecture. What I was trying to say is that we don’t need audio quality of 64bits.
Anyways, I’m not an expert in this field so I will shut up now. If you guys say that simply building x64 binaries is not possible, then I believe you. And that’s the answer to my original question.
Having said that, I would still love to see 64bit Renoise in 2010 ;)

I understood, i was trying to explain:if a 64-bit plugin expects his host to treat his 64-bit data that it submits, there is no way to force the plugin to submit 32-bit data instead.

same here; i didn’t believe that we need native 64 bit support for audio and plugins, we only need the adress-space :)