New Tool (3.0): ReQuest - Web server library for Renoise

Wait, what? Why?

Renoise’s scripting was a bit limited for what I wanted to do, so I got to thinking about how I could possibly augment it.

The problems:

• Renoise’s scripting API limited my ability to develop truly custom GUIs. Say, a piano roll maybe?
• Augmenting it with native code means compiling and debugging for 3 different platforms.
• Integrating with Python or some other runtime requires a third-party tool to be installed. Not cool.
• Building GUIs is pretty ugly in any language geared towards logic anyway.

So, is there anything around that solves these problems?
Something I can realistically count on being available for every computer?
Maybe even allowing for communication with mobile devices, if the mood’s right?

Yup, and you’re using it right now to read this post! ReQuest allows you to tap into the power of the user’s web browsers to extend Renoise using familiar, battle-hardened, dependable tech.

Well, what’s it got?

It’s packing:

• Simple routing, with parameterized URLs courtesy of Lua’s pattern matching.
• Support for custom middleware.
• Simple cookie interface.
• Simple HTML forms interface.
• Simple WebSockets interface.
• A clean, well-documented API.
• Plenty of example code to get you started.

How do I use it?

It’s a library, but it comes packaged with lots of pretty syntax-highlighted sample code. Just go to Tools >> Start Server after installing and it’ll automatically open your browser to ReQuest in action, and all the information you should need to use its code in your own tools.

Or if you’d prefer to check out some sample code here on the forums, click here.

Workflow-wise, the sky’s the limit. Using routing to send/receive data to/from Renoise via AJAX (with a custom REST API) is probably ideal where it can be used; lots of libraries exist to simplify AJAX requests.

WebSockets can be used for anything else.

That’s a really lame name.

Yeh, sorry. Apache was taken.

I’ll be poking my head in and out of the forums here, so feel free to ask me if you need help or have a feature ReQuest. (See what I did there? Har har har)

If I don’t respond quickly enough, feel free to hassle me on Twitter (@need12648430) until I bend to your will.

The updated version can be found here!

Bitwise ops in Lua are a ROYAL pain in the ass, but this is good tech for the job. So I’ll add it in the future.

I’m assuming you already know this, but just incase, Renoise extends Bitwise operators out-of-the-box by including this lib:

http://bitop.luajit.org/api.html

Also, I feel a bit like the Lua Librarian here but have you seen ActionServer(2010)?

Anyways, great project and implementation. I can see a lot of potential with the improvements in web browser tech in the last 5 years.

I’m assuming you already know this, but just incase, Renoise extends Bitwise operators out-of-the-box by including this lib:

http://bitop.luajit.org/api.html

Yeh, but I’m not used to bitwise ops being implemented as functions. I’m not particularly fond of the approach.

Also, I feel a bit like the Lua Librarian here but have you seen ActionServer(2010)?

Anyways, great project and implementation. I can see a lot of potential with the improvements in web browser tech in the last 5 years.

I hadn’t heard about that project, actually! Very interesting. Glad I’m not the only one who thought it’d be a good idea to put a web server in a DAW… Definitely reassuring. Aha.

Thanks! I see that same potential, I bet true synchronization would be possible over LAN with WebSockets. Polling isn’t the best approach for that. Hm.

1.0!

• multipart/form-data is now fully supported!
• WebSockets are now fully supported, with a clean interface to boot!
• API is restructured, allowing for more powerful routing chains!
• Cookie middleware added!
• Example code rewritten from scratch; install and go to Tools > Start Server to have a peak!
  Download

Can you give an example of usage of this tool?

Can you give an example of usage of this tool?

If you mean code, sure. This is all copied directly from the tool itself, just go to Tools >> Start Server to see it live.

Basics

Getting Started

– contains all the basic HTTP stuff
require(“request-http”)

– takes a host and port as arguments
local server = HTTPServer(“localhost”, 80)

– add a basic index page
server.router:get(“/”, function(request, response)
response:send_html(200, “Hello, world!”)

– let the router know the chain ended here; we already sent a response
return true
end)

– start the server
server:start()

– stop the server (once you’re done)
server:stop()

Parameterized Routing

– see Lua patterns for more information
server.router:get(“/(.*)”, function(request, response, params)
response:send_html(200, “The URL is: /” … params[1])

return true
end)

Forms

– this is our form
server.router:get(“/form”, function(request, response)
local content =
“<form method="post" action="/form">” …
“Content to POST:
” …
“<input type="text" name="message" /> <input type="submit" />” …
“”

response:send_html(200, content)

return true
end)

– and this is our processor for it
server.router:post(“/form”, function(request, response)
if request.form_data[“message”] ~= nil then
response:send_html(200, "You sent: " … request.form_data[“message”])
end

return true
end)

File Forms

– this is our form
server.router:get(“/form”, function(request, response)
local content =
“<form method="post" action="/form" enctype="multipart/form-data" >” …
“Content to POST:
” …
“<input type="file" name="files" multiple /> <input type="submit" />” …
“”

response:send_html(200, content)

return true
end)

– and this is our processor for it
server.router:post(“/form”, function(request, response)
local content = “”

for filename, contents in pairs(request.form_data[“files”]) do
local file = io.open (“uploads/” … filename, “wb”)
file:write(contents)
file:close()

content = content … filename … " uploaded!
"
end

response:send_html(200, content)

return true
end)

Middleware

Custom

server.router:use(function (request, response)
print(request.resource)

– note: it doesn’t return true, as it only does some processing
end)

– a pattern can also be specified
server.router:use(“/”, function (request, response)
print(request.resource)
end)

Packaged Middleware

– middleware is contained in the request-middleware module
require(“request-middleware”)

Static Files

– serves static files from a specific directory
– should probably be added LAST in the routing chain
– in case a matching file is not found, a callback should be specified

function file_not_found(request, response)
response:send_html(404, “No such file exists.”)

return true
end

server.router:use(static(“www”, file_not_found))

Cookies

– contains UTC timestamp() function useful for cookies
require(“request-utils”)

– parses and stores cookies in a newly-created request.cookies table
– also adds response:set_cookie(name, content, expiry)
server.router:use(cookies())

server.router:get(“/cookie”, function (request, response)
local content = “”

if request.cookies[“message”] ~= nil then
content = content … "Cookie contains: " … request.cookies[“message”] … “<br /<”
end

content = content …
“<form method="post" action="/cookie">” …
“Set cookie to:
” …
“<input type="text" name="message" /> <input type="submit" />” …
“”

response:send_html(200, content)

return true
end)

server.router:post(“/cookie”, function (request, response)
if request.form_data[“message”] ~= nil then
response:set_cookie(“message”, request.form_data[“message”], timestamp() + 60)
end

response:send_html(200, “Cookie set!”)

return true
end)

WebSockets

– adds request:websocket_requested()
– also adds response:open_websocket(handler) which returns a WebSocketClient instance
– see request-websocket-utils for more info on WebSocketClient
server.router:use(websockets())

server.router:get(“/websocket”, function(request, response)
– a basic echo server handler
local echo_server = {
[“on_open”] =
function(websocket)
print(websocket.id, “connected! :)”)
end,
[“on_message”] =
function(websocket, opcode, message)
– first argument is the FIN bit
– it indicates that this message is complete - not fragmented
websocket:send(1, opcode, message)
end,
[“on_close”] =
function(websocket)
print(websocket.id, “disconnected! :(”)
end
}

if request:websocket_requested() then
local client = response:open_websocket(echo_server)
else
response:send_html(403, “WebSocket clients only.”)
end

return true
end)

If you mean “how can this be used to make neat tools,” then here are some examples:

• You can combine it with an OSC server to send note ons and note offs from HTML5 canvas (neat procedural music).
• You can develop a protocol on top of WebSockets to allow real-time editing from multiple computers; even phones.
• That can also be used remotely for collaborative work, though there will be latency outside of your LAN.
• HTML5 isn’t restricting about what you can do, GUI wise. A formal piano roll is entirely possible with this library.
• Not limited to this library, but with Renoise’s delay column it’s likely possible to support different BPMs for each track.
  You’d need a strong GUI to keep that usable though.

Just some thoughts. I’ll probably be taking on a few myself after a short break.

Thank you very much.
I was curious about these neat tools mainly.

i will study it sometime if it is way how i can program my own sequencer in jquery

Thank you very much.
I was curious about these neat tools mainly.

i will study it sometime if it is way how i can program my own sequencer in jquery

Good luck! I’d love to see what you do with it, so keep me posted.

I kept coming back to this tool, curious as to what it would be used for. Then I read a bit more and started to grok it. Kudos, good sir. This seems awesome. I hope to see some badass tools using this.

Has anyone ever used this?

Not that I know of, no. I hope the author has benefited from it in the ways he described.

I know I’ve mentioned it, because

A formal piano roll is entirely possible with this library.

This is what it would take. Well, this, and my “voice-runner” class working their magic together.

Could I just mention that, yes, a simple piano roll is possible doing it via web sockets etc… But I personally wouldn’t go about it this way. I would do it by just writing a C library of the ‘piano roll front end’ and importing that into Renoise via lua. Should be more efficient, albeit you would have to compile the library for the different operating systems

@4tey: yes, but who would do such a thing?

https://forum.renoise.com/t/experimental-miniroll/40559

Haha. Ok. Well, it’s a heck of work to make the mouse interactions and everything work as they should in pure C.

High-level languages, on the other hand… they are getting faster, and development tools are improving all the time.

One project that I feel is representing this shift is Electron - a platform based on node.js, which makes it a breeze to support all OS’es.

Developers, developers…

@4tey: yes, but who would do such a thing?

Oh, not me sir…maybe someone else who goes by the name 4Tey Yes MiniRoll is network socket driven, but when I was messing around with ‘sockets’ I happened to get some data from Renoise externally, so I thought wouldn’t it be nice to plot the information in a piano roll format. But after all that in the back of my mind I knew I could make it way more efficient by just directly incorporating a C library of ‘MiniRoll’ (using say GTK toolkit for the mouse/buttons events etc…) into Renoise lua

Sure high level languages are indeed useful danoise. As computers get ‘faster’ it feasible to program at a higher level. I admit I do tend to gravitate towards lower level programming though. Funnily enough I wouldn’t enjoy using your vLib for example danoise. Not because it isn’t clever and well done on your part, but because if I did program say a small tool with a ‘helper library’ I wouldn’t feel as though I’ve fully appreciated my program that I’ve written AND it could just be executing less efficiently. I’m not much of a programmer anyway danoise, I literally just hack my way through things. Just my preference to use straight C rather than say C# or Python or Haskell or Electron

Also, with that new firefox engine that has been announced (massive speed improvements), perhaps webapps can finally become THE GUI platform? (the way it was hyped as about 5 years ago)

On the other hand, other DAWs (i e “Reaper”), haven’t done anything special yet to go in this direction (?). And if renoise.Viewbuilder was improved/replaced, the current API could be damn good for anything (already is, unless you’re limited by the GUI capabilities).

Cool, I see my necro-bump has got the scripters talking B) .

I guess this project is a more pimped out version of Bantai’s ‘never finished’ ‘action server’ project? (http://forum.renoise.com/index.php/topic/26964-actionserver-an-internal-web-server/ ).

Would be cool to have some kind of browser like control for use in a tablet controlling renoise.

Woah. This has been pretty active again. I almost forgot I wrote this.

I should probably do something with it.

This thing can probably be used together with Electron? I’m no coder so I thought I’d throw this idea here in case anyone fees inspired trying it out.

Electorn is a framework for coding GUI:s with javascript (chrome backbone) and easily deploy them as native applications for windows, macos and linux.

Opening such an app with io.popen from Renoise, I could see that Viewbuilder could be substituted with a much more flexible electron GUI, even acting as something that appears well integrated to Renoise.

EDIT: I see now that it’s already been mentioned

Hello,

in Renoise 3.1.1 whe i try to start server, i get:

./request-http.lua:21: variable 'socket_error' is not declared
stack traceback:
  [C]: in function '_error'
  [string "local mt = getmetatable(_G)..."]:29: in function <[string "local mt = getmetatable(_G)..."]:24>
  ./request-http.lua:21: in function <./request-http.lua:9>
  [C]: in function 'HTTPServer'
  main.lua:117: in function 'start'
  main.lua:9: in function <main.lua:9>

any help ?