Jump to content


Photo

New Tool (3.0): ReQuest - Web server library for Renoise

http tool request web browser bridge

  • Please log in to reply
17 replies to this topic

#1 0x6a61686e

0x6a61686e

    New Member

  • Normal Members
  • Pip
  • 6 posts

Posted 09 March 2015 - 05:12

Wait, what? Why?
 
Renoise's scripting was a bit limited for what I wanted to do, so I got to thinking about how I could possibly augment it.
 
The problems:

  • Renoise's scripting API limited my ability to develop truly custom GUIs. Say, a piano roll maybe?
  • Augmenting it with native code means compiling and debugging for 3 different platforms.
  • Integrating with Python or some other runtime requires a third-party tool to be installed. Not cool.
  • Building GUIs is pretty ugly in any language geared towards logic anyway.

So, is there anything around that solves these problems?
Something I can realistically count on being available for every computer?
Maybe even allowing for communication with mobile devices, if the mood's right?
 
Yup, and you're using it right now to read this post! ReQuest allows you to tap into the power of the user's web browsers to extend Renoise using familiar, battle-hardened, dependable tech.
 
Well, what's it got?
 
It's packing:

  • Simple routing, with parameterized URLs courtesy of Lua's pattern matching.
  • Support for custom middleware.
  • Simple cookie interface.
  • Simple HTML forms interface.
  • Simple WebSockets interface.
  • A clean, well-documented API.
  • Plenty of example code to get you started.

How do I use it?
 
It's a library, but it comes packaged with lots of pretty syntax-highlighted sample code. Just go to Tools >> Start Server after installing and it'll automatically open your browser to ReQuest in action, and all the information you should need to use its code in your own tools.

 

Or if you'd prefer to check out some sample code here on the forums, click here.

 

Workflow-wise, the sky's the limit. Using routing to send/receive data to/from Renoise via AJAX (with a custom REST API) is probably ideal where it can be used; lots of libraries exist to simplify AJAX requests.

 

WebSockets can be used for anything else.
 
That's a really lame name.
 
Yeh, sorry. Apache was taken.

I'll be poking my head in and out of the forums here, so feel free to ask me if you need help or have a feature ReQuest. (See what I did there? Har har har)
 
If I don't respond quickly enough, feel free to hassle me on Twitter (@need12648430) until I bend to your will.
  
The updated version can be found here!


Edited by 0x6a61686e, 16 March 2015 - 15:58.
Removed outdated attachment.

  • Conner_Bw, afta8, re.dread and 4 others like this

#2 Conner_Bw

Conner_Bw

    Probably More God or Borg Than Human Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 7059 posts
  • Gender:Male
  • Location:Montreal, Quebec, Canada

Posted 09 March 2015 - 22:02

Bitwise ops in Lua are a ROYAL pain in the ass, but this is good tech for the job. So I'll add it in the future.

 

I'm assuming you already know this, but just incase, Renoise extends Bitwise operators out-of-the-box by including this lib:

 

http://bitop.luajit.org/api.html


cpu Lenovo X220, Intel i7-2640M @ 2.80GHz ×4 os Windows 10 / Ubuntu 16.04 LTS
My Homepage » : My Renoise Tools » : Normalize Your Sig » : One million clicks! »


#3 Conner_Bw

Conner_Bw

    Probably More God or Borg Than Human Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 7059 posts
  • Gender:Male
  • Location:Montreal, Quebec, Canada

Posted 10 March 2015 - 00:36

Also, I feel a bit like the Lua Librarian here but have you seen ActionServer (2010)?

 

Anyways, great project and implementation. I can see a lot of potential with the improvements in web browser tech in the last 5 years.


cpu Lenovo X220, Intel i7-2640M @ 2.80GHz ×4 os Windows 10 / Ubuntu 16.04 LTS
My Homepage » : My Renoise Tools » : Normalize Your Sig » : One million clicks! »


#4 0x6a61686e

0x6a61686e

    New Member

  • Normal Members
  • Pip
  • 6 posts

Posted 10 March 2015 - 02:48

I'm assuming you already know this, but just incase, Renoise extends Bitwise operators out-of-the-box by including this lib:

 

http://bitop.luajit.org/api.html

Yeh, but I'm not used to bitwise ops being implemented as functions. I'm not particularly fond of the approach.  :unsure:

 

 

Also, I feel a bit like the Lua Librarian here but have you seen ActionServer (2010)?

 

Anyways, great project and implementation. I can see a lot of potential with the improvements in web browser tech in the last 5 years.

I hadn't heard about that project, actually! Very interesting. Glad I'm not the only one who thought it'd be a good idea to put a web server in a DAW... Definitely reassuring. Aha.

 

Thanks! I see that same potential, I bet true synchronization would be possible over LAN with WebSockets. Polling isn't the best approach for that. Hm.



#5 0x6a61686e

0x6a61686e

    New Member

  • Normal Members
  • Pip
  • 6 posts

Posted 16 March 2015 - 00:44

1.0!
  • multipart/form-data is now fully supported!
  • WebSockets are now fully supported, with a clean interface to boot!
  • API is restructured, allowing for more powerful routing chains!
  • Cookie middleware added!
  • Example code rewritten from scratch; install and go to Tools > Start Server to have a peak!
Download

Edited by dblue, 16 March 2015 - 13:58.
Removed attachment. Linked to official Tools page instead.


#6 Akiz

Akiz

    Guruh Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPip
  • 966 posts
  • Gender:Male

Posted 16 March 2015 - 13:06

Can you give an example of usage of this tool?



#7 0x6a61686e

0x6a61686e

    New Member

  • Normal Members
  • Pip
  • 6 posts

Posted 16 March 2015 - 15:45

Can you give an example of usage of this tool?

 

If you mean code, sure. This is all copied directly from the tool itself, just go to Tools >> Start Server to see it live.

 

Basics

----------------------------------------------------------------------------------------------------

Getting Started

-- contains all the basic HTTP stuff
require("request-http")

-- takes a host and port as arguments
local server = HTTPServer("localhost", 80)

-- add a basic index page
server.router:get("/", function(request, response)
    response:send_html(200, "Hello, world!")
    
    -- let the router know the chain ended here; we already sent a response
    return true
end)

-- start the server
server:start()

-- stop the server (once you're done)
server:stop()

 

Parameterized Routing

-- see Lua patterns for more information
server.router:get("/(.*)", function(request, response, params)
    response:send_html(200, "The URL is: /" .. params[1])
    
    return true
end)

 

Forms

-- this is our form
server.router:get("/form", function(request, response)
    local content =
        "<form method=\"post\" action=\"/form\">" ..
        "Content to POST: <br />" ..
        "<input type=\"text\" name=\"message\" /> <input type=\"submit\" />" ..
        "</form>"
    
    response:send_html(200, content)
    
    return true
end)

-- and this is our processor for it
server.router:post("/form", function(request, response)
    if request.form_data["message"] ~= nil then
        response:send_html(200, "You sent: " .. request.form_data["message"])
    end
    
    return true
end)

 

File Forms

-- this is our form
server.router:get("/form", function(request, response)
    local content =
        "<form method=\"post\" action=\"/form\" enctype=\"multipart/form-data\" >" ..
        "Content to POST: <br />" ..
        "<input type=\"file\" name=\"files\" multiple /> <input type=\"submit\" />" ..
        "</form>"
    
    response:send_html(200, content)
    
    return true
end)

-- and this is our processor for it
server.router:post("/form", function(request, response)
    local content = ""
    
    for filename, contents in pairs(request.form_data["files"]) do
        local file = io.open ("uploads/" .. filename, "wb")
        file:write(contents)
        file:close()
        
        content = content .. filename .. " uploaded!<br />"
    end
    
    response:send_html(200, content)
    
    return true
end)

 

Middleware

----------------------------------------------------------------------------------------------------

 

Custom

server.router:use(function (request, response)
    print(request.resource)
    
    -- note: it doesn't return true, as it only does some processing
end)

-- a pattern can also be specified
server.router:use("/", function (request, response)
    print(request.resource)
end)

 

Packaged Middleware

-- middleware is contained in the request-middleware module
require("request-middleware")

 

Static Files

-- serves static files from a specific directory
-- should probably be added LAST in the routing chain
-- in case a matching file is not found, a callback should be specified

function file_not_found(request, response)
    response:send_html(404, "No such file exists.")
    
    return true
end

server.router:use(static("www", file_not_found))

 

Cookies

-- contains UTC timestamp() function useful for cookies
require("request-utils")

-- parses and stores cookies in a newly-created request.cookies table
-- also adds response:set_cookie(name, content, expiry)
server.router:use(cookies())

server.router:get("/cookie", function (request, response)
    local content = ""
    
    if request.cookies["message"] ~= nil then
        content = content .. "Cookie contains: " .. request.cookies["message"] .. "<br /<"
    end
    
    content = content ..
        "<form method=\"post\" action=\"/cookie\">" ..
        "Set cookie to: <br />" ..
        "<input type=\"text\" name=\"message\" /> <input type=\"submit\" />" ..
        "</form>"
    
    response:send_html(200, content)
    
    return true
end)

server.router:post("/cookie", function (request, response)
    if request.form_data["message"] ~= nil then
        response:set_cookie("message", request.form_data["message"], timestamp() + 60)
    end
    
    response:send_html(200, "Cookie set!")
    
    return true
end)

 

WebSockets

-- adds request:websocket_requested()
-- also adds response:open_websocket(handler) which returns a WebSocketClient instance
-- see request-websocket-utils for more info on WebSocketClient
server.router:use(websockets())

server.router:get("/websocket", function(request, response)
    -- a basic echo server handler
    local echo_server = {
        ["on_open"] =
            function(websocket)
                print(websocket.id, "connected! :)")
            end,
        ["on_message"] =
            function(websocket, opcode, message)
                -- first argument is the FIN bit
                -- it indicates that this message is complete - not fragmented
                websocket:send(1, opcode, message)
            end,
        ["on_close"] =
            function(websocket)
                print(websocket.id, "disconnected! :(")
            end
    }
    
    if request:websocket_requested() then
        local client = response:open_websocket(echo_server)
    else
        response:send_html(403, "WebSocket clients only.")
    end
    
    return true
end)

 

----------------------------------------------------------------------------------------------------

 

If you mean "how can this be used to make neat tools," then here are some examples:

  • You can combine it with an OSC server to send note ons and note offs from HTML5 canvas (neat procedural music).
  • You can develop a protocol on top of WebSockets to allow real-time editing from multiple computers; even phones.
  • That can also be used remotely for collaborative work, though there will be latency outside of your LAN.
  • HTML5 isn't restricting about what you can do, GUI wise. A formal piano roll is entirely possible with this library.
  • Not limited to this library, but with Renoise's delay column it's likely possible to support different BPMs for each track.
    You'd need a strong GUI to keep that usable though.

Just some thoughts. I'll probably be taking on a few myself after a short break.


Edited by 0x6a61686e, 16 March 2015 - 16:04.

  • Djeroek likes this

#8 Akiz

Akiz

    Guruh Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPip
  • 966 posts
  • Gender:Male

Posted 16 March 2015 - 16:17

Thank you very much.
I was curious about these neat tools mainly.

i will study it sometime if it is way how i can program my own sequencer in jquery :-)



#9 0x6a61686e

0x6a61686e

    New Member

  • Normal Members
  • Pip
  • 6 posts

Posted 16 March 2015 - 16:25

Thank you very much.
I was curious about these neat tools mainly.

i will study it sometime if it is way how i can program my own sequencer in jquery :-)

 

Good luck! I'd love to see what you do with it, so keep me posted. :)



#10 wahrk

wahrk

    Big GrandDaddy Member

  • Normal Members
  • PipPipPipPipPipPipPipPip
  • 487 posts
  • Gender:Male
  • Location:Austin, Tx
  • Interests:music, computers, language, philosophy, games

Posted 22 August 2015 - 05:10

I kept coming back to this tool, curious as to what it would be used for. Then I read a bit more and started to grok it. Kudos, good sir. This seems awesome. I hope to see some badass tools using this.



#11 Djeroek

Djeroek

    Probably More God or Borg Than Human Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6692 posts
  • Gender:Male
  • Location:Borneo

Posted 04 November 2016 - 11:39

Has anyone ever used this?


  • Neurogami and Akiz like this

#12 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6299 posts
  • Gender:Male
  • Interests:wildlife + urban trekking

Posted 05 November 2016 - 12:38

Not that I know of, no. I hope the author has benefited from it in the ways he described. 

I know I've mentioned it, because
 

A formal piano roll is entirely possible with this library.

 

This is what it would take. Well, this, and my "voice-runner" class working their magic together. 

 


Tracking with Stuff. API wishlist | Soundcloud


#13 4Tey

4Tey

    Chief Above Chief Member

  • Normal Members
  • PipPipPipPipPipPip
  • 377 posts
  • Gender:Male

Posted 05 November 2016 - 16:12

Could I just mention that, yes, a simple piano roll is possible doing it via web sockets etc..  But I personally wouldn't go about it this way.  I would do it by just writing a C library of the 'piano roll front end' and importing that into Renoise via lua.  Should be more efficient, albeit you would have to compile the library for the different operating systems :)



#14 danoise

danoise

    Probably More God or Borg Than Human Member

  • Renoise Team
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6299 posts
  • Gender:Male
  • Interests:wildlife + urban trekking

Posted 05 November 2016 - 17:47

@4tey: yes, but who would do such a thing? 

http://forum.renoise...ental-miniroll/

 

Haha. Ok. Well, it's a heck of work to make the mouse interactions and everything work as they should in pure C. 

 

High-level languages, on the other hand... they are getting faster, and development tools are improving all the time. 

One project that I feel is representing this shift is Electron - a platform based on node.js, which makes it a breeze to support all OS'es. 

 

Developers, developers...


Tracking with Stuff. API wishlist | Soundcloud


#15 4Tey

4Tey

    Chief Above Chief Member

  • Normal Members
  • PipPipPipPipPipPip
  • 377 posts
  • Gender:Male

Posted 05 November 2016 - 19:08

@4tey: yes, but who would do such a thing?

Oh, not me sir..maybe someone else who goes by the name 4Tey ;)  Yes MiniRoll is network socket driven, but when I was messing around with 'sockets' I happened to get some data from Renoise externally, so I thought wouldn't it be nice to plot the information in a piano roll format.  But after all that in the back of my mind I knew I could make it way more efficient by just directly incorporating a C library of 'MiniRoll' (using say GTK toolkit for the mouse/buttons events etc..) into Renoise lua :)

 

Sure high level languages are indeed useful danoise.  As computers get 'faster' it feasible to program at a higher level.  I admit I do tend to gravitate towards lower level programming though.  Funnily enough I wouldn't enjoy using your vLib for example danoise.  Not because it isn't clever and well done on your part, but because if I did program say a small tool with a 'helper library' I wouldn't feel as though I've fully appreciated my program that I've written AND it could just be executing less efficiently.  I'm not much of a programmer anyway danoise, I literally just hack my way through things.  Just my preference to use straight C rather than say C# or Python or Haskell or Electron :)



#16 joule

joule

    Guruh Motha Fakka is Levitating and Knows Everything About Renoise Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPip
  • 1410 posts
  • Gender:Not Telling
  • Location:Sweden
  • Interests:music, philosophy, engineering

Posted 05 November 2016 - 19:38

Also, with that new firefox engine that has been announced (massive speed improvements), perhaps webapps can finally become THE GUI platform? (the way it was hyped as about 5 years ago)

 

On the other hand, other DAWs (i e "Reaper"), haven't done anything special yet to go in this direction (?). And if renoise.Viewbuilder was improved/replaced, the current API could be damn good for anything (already is, unless you're limited by the GUI capabilities).



#17 Djeroek

Djeroek

    Probably More God or Borg Than Human Member

  • Normal Members
  • PipPipPipPipPipPipPipPipPipPipPipPipPipPipPip
  • 6692 posts
  • Gender:Male
  • Location:Borneo

Posted 05 November 2016 - 19:49

Cool, I see my necro-bump has got the scripters talking  B) .

 

I guess this project is a more pimped out version of Bantai's 'never finished' 'action server' project? ( http://forum.renoise...nal-web-server/ ).

 

Would be cool to have some kind of browser like control for use in a tablet controlling renoise.



#18 0x6a61686e

0x6a61686e

    New Member

  • Normal Members
  • Pip
  • 6 posts

Posted 07 November 2016 - 04:42

Woah. This has been pretty active again. I almost forgot I wrote this.

 

I should probably do something with it. :smashed:


Edited by 0x6a61686e, 07 November 2016 - 04:43.

  • danoise likes this





Also tagged with one or more of these keywords: http, tool, request, web browser bridge