OSC latency in a lua script

Hey there ! I’m currently experimenting with bridging Renoise with Tidal Cycles. The goal is to do the sequencing with Tidal and the sound generation with Renoise.

I’ve tried two approaches: make Tidal send OSC messages that Renoise can understand out of the box, or make Renoise understand the messages Tidal sends to the regular audio backend.

The first approach works quite well, but introduces some warts in Tidal usage.

The second approach looks promising, but I have some timing issues. Basically I tried this in the TestPad:

  1. tidal sends a message to port 57120
  2. my “redirt” script listens on 57120 and receive the message
  3. the message is analysed, (instrument, track, note, velocity) are extracted, and a /renoise/trigger/note_on message is sent back to renoise on port 8000 (note triggering is not exposed to the lua API).

Now, when I do this the timing sucks. Like very very much. And I have questions:

  • Is it unavoidable?
  • Is it because the TestPad does not have some form of optimization?
  • My OSC messages have timetags, so you can send a message in advance and have the target schedule it exactly at the right moment. Does renoise do that?

Here is my proof of concept, for reference.

local OscMessage = renoise.Osc.Message
local OscBundle = renoise.Osc.Bundle

-- open a socket connection to the server
local server, socket_error = renoise.Socket.create_server(
  "localhost", 57120, renoise.Socket.PROTOCOL_UDP)
   
if (socket_error) then 
  renoise.app():show_warning(("Failed to start the " .. 
    "OSC server. Error: '%s'"):format(socket_error))
  return
end

-- open a socket connection to the client
local client, socket_error = renoise.Socket.create_client(
  "localhost", 8000,
  renoise.Socket.PROTOCOL_UDP)

if (socket_error) then 
  renoise.app():show_warning(("Failed to open the osc loopback with error: '%s'"):format(socket_error))
  return
end

function linlin(input, srclo, srchi, dstlo, dsthi)
  -- clamp input
  if input < srclo then input = srclo end
  if input > srchi then input = srchi end
  -- normalize over [0,1]
  local norm = (input - srclo) / srchi
  -- transpose to [dstlo,dsthi]
  return dstlo + norm * (dsthi-dstlo)
end

-- parse a tidal bundle and return a list of messages
function parseTidal(bundle)
  local msgs = {}
  for _,element in ipairs(bundle.elements) do
    if element.pattern == "/play2" then
      local msg = {}
      local name = ""
      for i,argument in ipairs(element.arguments) do       
        if i%2 == 1 then
          name = argument.value
        else
          msg[name] = argument.value
        end
      end
      msgs[#msgs+1] = msg
    end
  end
  return msgs
end

function runTidal(msg)
  -- rprint(msg)
  -- track number
  local track = msg.orbit+1
  -- should we trigger a sound ?
  if msg.s then
    -- select instrument by number (or the first one if something is wrong)
    local instr = (tonumber(msg.s) or 0) + 1
    -- default octave is 4
    local octave = msg.octave or 4
    local note = octave * 12 + msg.note
    -- rescale velocity from [0,1] to [0,127]
    local velocity = linlin(msg.gain or 1 , 0, 1, 0, 127)
    
    client:send(OscMessage("/renoise/trigger/note_on", {
      {tag="i", value=instr},
      {tag="i", value=track},
      {tag="i", value=note},
      {tag="i", value=velocity}
    }))
  end
end

server:run {
  socket_message = function(socket, data)
    -- decode the data to Osc
    local mob, osc_error = renoise.Osc.from_binary_data(data)
    
    -- show what we've got
    if (mob) then
      local msgs = parseTidal(mob)
      for _,msg in ipairs(msgs) do
        runTidal(msg)
      end
    else
      print(("Got invalid OSC data, or data which is not " .. 
        "OSC data at all. Error: '%s'"):format(osc_error))
    end      
  end    
}
1 Like

Hi. This is inpreventable in making digital music.

  • Every (thing) in the line generates latency.

(user interface, network, sound plugin, sound card)

  • Setting your latency in Renoise (and other things) to as low as possible will help.
    (image at the bottom)*1

Calculating a lot of instruments requires latency…
Only (a few) simple instruments will run on low latency.

  • 20 msec of latency is the shortest time that can be defined, heard, by the ear.
    Anything more, after that 20 msec, will make you experience lag.
    (source: Algemene Muziekleer, Theo Willemze, music theory book)

Anything in your flow (your script too) will generate latency. So, they all need to be set to minimal latency to experience less than 20 msec to not experience lag.

  • You can view the latency of each plugin in Renoise in the
    menu: Song… Plugin Delay Compensation Info…

*1 Preferences - setting audio latency