It seems like the Osc.Message has a 4k buffer :
local OscMessage = renoise.Osc.Message
local oscdata = {}
for i = 1,819 do
table.insert(oscdata,{tag="i",value=i})
end
local msg = OscMessage("/bf",oscdata)
which generates an exception ‘out of buffer memory’
any chance this buffer can be made bigger (a lot bigger) or unlimited? 
That would probably cost speed performance which is not something that is desirable in the audio world…
Is there any specific reason why these messages cannot be chopped into smaller chunks?
No, of course everything can be chopped up, it just pushes some complexity up to the application using OSC. For example im writing some code to request pattern data via OSC and according to the OSC protocol an int is 32 bits, but each needs a type tag so you effectively have 5 bytes per int, i.e. a minimum of 10 bytes per note (disregarding effect columns and such). So within 4k you have enough space for about 400 notes. If your pattern has 64 lines that gives you 6.25 (400 / 64) fully populated tracks or note columns.
So with a ‘naive’ / basic implementation of a system to serialize pattern data you’ll hit the buffer limit right away, unless you start dealing with empty lines and note columns by some mechanism. This is the stage im at now where im using -1 to signify an empty line and -2 to denote the end of the note columns in that line. Even with this scheme i’ll probably hit the buffer limit with densely populated patterns and also have to add some kind of continuation scheme, i.e. flag the message as ‘incomplete’ so the client knows to expect more data which is can compose when it arrives. This implies my server code has to monitor the construction of OSC Packet data to make sure im within the buffer limit and queue up any overflow data for extra messages. So as you can see the level of complexity I have to deal with starts to increase. Now, being a ‘lazy programmer’ I would rather just serialize the whole thing, empty data and all and not worry about it too much 
I appreciate the constraints under which the tools have to run to allow the audio engine time to do its work, but it would be nice if I could choose to use larger buffers knowing the possible side-effects, even if there’s no guarantee that the message will be serialised within the next available time slice. After all one benefit of using TCP as the transport layer is that you dont have to send all the data at once. 
as a side note the OSC Spec page says this :
but i’ve noticed that neither the HTTP OSC client and server seem to do this. Im not sure how strict the should is in their spec, but having the size available really helps to make the packet decoder implementation easier. If you know the size of the packet you just buffer the data until you have enough to match the length header then pass the bytes off to the decoder, but without knowing the size the decoder has to be built as some kind of state machine and be involved in buffer management (either returning unused bytes or pulling bytes as they are available and notifying when its got a complete packet).
thanks for listening / reading 
Sounds more like a snippet from the TCP or UDP hardware layer descriptions to me than specific OSC specifications. Probably added for extra information.
But i suspect to get optimal and speedy performance with OSC messaging, small buffer-sizes still matter for the same as with audio cards perform faster when having small buffersizes. Does require more cpu consumption though in the latter case.
maybe the buffer could be variable on the tcp/streaming side?
well, after thinking about it some more I realised that it was a waste to use a whole 32-bit int for an 8-bit value, so im packing the instrument and note value into one int and i can pack the 3 standard fx columns into another int so I use 10 bytes rather than 25 for that data. As always constraints inspire creativity.
still wouldnt mind bigger buffers though 
We used to have to deal with a maximum ram of 64K once… (of which Bill Gates said that it should be enough for everyone) leaving the programmer with only 40K of useful RAM left after the OS snooped off 20K from it
In that regard, having chunks of 4K available is quite a lot!
Takik’s opinion required, as this sounds a lot like bullshit to me.
not to derail the thread too much but he denied saying that, it was about 640k not 64k AND it was in 1981. 4k isn’t a lot in 2010.
Also I realised that the point about processing time isnt relevant considering that you can create as many messages as you like, and in fact there will probably be more overhead when creating multiple messages rather than using one larger buffer.
anyway, it would be good to get some feedback from a developer before I worry too much about byte-packing my data.
That was ofcourse my bullshit contribution.
Ofcourse but that depends on Taktik indeed. If you can generate 64K buffered messages, should pose no problems in sending 4K message either if there isn’t any more data in it.
If the size limit is indeed programmable and not limited by the Lua library or worse:hardware layer for instance.
I know UDP datagram packages limit to approx. 64Kbytes (some bytes are lost to IP header and other information) , so assembling larger buffers than this size doesn’t seem efficient if i understood the UDP protocol definition correctly.
And there is probably also a reason for adapting buffering recovering lost packages…
But as you suspect, smaller packages may probably cause more overhead when more packages get lost during the transfer. There is a balance between optimal sized packages and error correction.
Socket buffer sizes are platform dependent and can also be changed during runtime. On my windows here its 8192 bytes.
And an OSC message has to fit into a socket buffer. (Well, in theory it would be possible to split the messages into multiple socket messages, but I don’t think many Osc implementation will support this. At least we don’t.)
I’ve used 4096, cause this seems to be a good min socket buffer guess. So if the OSC message fits into this size, its very very likely that you also will be able to send and receive it on many platforms, setups.
Note that when receiving OSC messages there is no such limit in Renoise. There we simply uses the buffers we got send from the clients with no size limit.
Will nevertheless do some research how other Osc impls deal with the problem.
mdk : Curious what you are working on, btw
My initial mini-project is to build a drum editor in flash, kind of like the cubase drum editor, but with some useful features i’ve thought of over the years or have seen in other stuff like Guru and various reaktor ensembles. Not because im averse to using the keyboard and the renoise UI but after spending all day programming its sometimes nice to just sit back and click.
As part of that I want to create an API via OSC to access and manipulate whatever there is available in renoise, essentially treating renoise as a music engine which I can control via whatever I have that can talk OSC, flash, android devices etc…
So the first stage is to read and write pattern data. I’ve got my /renoise/pattern/track/get (pattern_index,track_index) method working on both sides, renoise and flash. Im just writing the /renoise/notes/set (pattern_index,track_index,[note_data,…]) part now and the first pass prototype UI is almost ready.