Extracting values from xml files

Renoise Tool Development
1

I’m a bit of an xml noob and was wondering if there is an easy way to get values from nodes in an xml file.

For example in the following:

<?xml version="1.0" encoding="UTF-8"?>
<FilterDeviceClipboard doc_version="0">
  <DeviceSlot type="GainerDevice">
    <IsMaximized>true</IsMaximized>
    <IsSelected>true</IsSelected>
    <SelectedPresetName>Init</SelectedPresetName>
    <SelectedPresetIsModified>false</SelectedPresetIsModified>
    <IsActive>
      <Value>1.0</Value>
      <Visualization>Device only</Visualization>
    </IsActive>
    <Volume>
      <Value>1.0</Value>
      <Visualization>Mixer and Device</Visualization>
    </Volume>
    <Panning>
      <Value>0.5</Value>
      <Visualization>Device only</Visualization>
    </Panning>
    <LPhaseInvert>false</LPhaseInvert>
    <RPhaseInvert>false</RPhaseInvert>
    <SmoothParameterChanges>true</SmoothParameterChanges>
  </DeviceSlot>
</FilterDeviceClipboard>

Say if I want to get the Panning value is there a way to do this via the API or do I have to do a load of string matching using the string library?

I’m actually trying to extract some data from Ableton live song files where the xml structure is way more complex, I can explore the tree using an xml viewer so was hoping there is an easy way to pull key bits of data via lua.

Thanks

2

I have this same question. How to obtain data from XML files?

For example, in a tool is the manifest.xml file, which is in the root folder of the tool:

  1. How to obtain the root address of the tool folder?
  2. How to extract the data from the XML file?

Actually, my case is particular. In a tool, inside a folder, I have an html file. I want to be able to open the html file in a browser from a button, using: renoise.app():open_url( url )

I use this function to obtain the path:

local function pht_user_guide_path()
  local data = os.getenv("APPDATA")
  local version = renoise.RENOISE_VERSION
  return "file:///"..data.."/Renoise/V"..version.."/Scripts/Tools/ID_OF_TOOL.xrnx/user_guide/NAME_OF_TOOL_user_guide_en.html"
end

The ID_OF_TOOL is in the file manifest.xml. This works with windows, but I do not like how it’s done.Is there any better way to do all this?

The manifest.xml file of the tool:

<?xml version="1.0" encoding="UTF-8"?>
<RenoiseScriptingTool doc_version="0">
  <ApiVersion>5</ApiVersion>
  <AutoUpgraded>false</AutoUpgraded>
  <Name>NAME</Name>
  <Id>ulneiz.NAME_v1.0</Id>
  <Version>1.00</Version>
  <Author>ulneiz</Author>
  <Category>NAME</Category>
  <Description></Description>
</RenoiseScriptingTool>
3

Say if I want to get the Panning value (…)

Some useful reading: http://lua-users.org/wiki/LuaXml

A pretty good and compact native LuaXML parser: https://github.com/manoelcampos/xml2lua

A quick example:

-- Load xml2lua
require("xml2lua")

-- Create a handler that converts the XML to a Lua table
local handler = require("xmlhandler.tree")

-- Instantiate XML parser and do some work
local parser = xml2lua.parser(handler)

-- Grab some example XML data from a TrackMixerDevice
local xml = renoise.song():track(1):device(1).active_preset_data

-- Parse that dang XML!
parser:parse(xml)

-- Get the TrackMixerDevice panning value as a string
local pan_string = handler.root.FilterDeviceClipboard.DeviceSlot.Panning.Value

-- Convert it to a number and do something cool with it
local pan_value = tonumber(pan_string)
print(string.format("The kickass panning value is %f dude!", pan_value))

PS: Please be mindful and careful of what your tool is actually doing here. Fair enough, the Config.xml is technically part of Renoise, but when a Renoise tool starts to fiddle about with files beyond its own local scope, there’s potential to drift into some pretty weird and sketchy territory regarding user privacy, permissions, etc.

4

This is great, do you know a library providing the exact reverse way, too, like lua table->xml ? I am still looking for a lua table<->json lib, which supports recursion/endless depth. I would be too lazy to implement myself, since I never use LUA outside Renoise.

EDIT: I am not sure if your linked xml2lua supports more depth than level 2? Couldn’t find a hint in the parser section for recursion, only when printing tables or so…

5

Thanks @dblue, very useful the way you have broken it down there. It’s been a few years since I posted my original query, currently I am in the same place as @ffx also looking for ways to go the other way. I have used this json library beforehttp://dkolf.de/src/dkjson-lua.fsl/home and it works well but from the homepage it states json has limitations that won’t support all lua tables.

you should be aware that not every Lua table can be represented by the JSON standard. For example tables that contain both string keys and an array part cannot be exactly represented by JSON. You can solve this by putting your array data in an explicit subtable.

So maybe XML is a better bet? My main use case for this is saving tool presets into xml or json files. I have looked at the renoise.Document.create() function which is used in tool preferences and seems to create xml files that can also be read from but I haven’t yet explored this in more detail… my question therefore is if there is an XML parser built into the API using the Document features?

And finally I wonder if it might just be easier to use lua written to a file to store lua tables? Something like this:http://lua-users.org/wiki/SaveTableToFile

6

FYI: I used this JSON libin gui automation tool, it only doesn’t support unlimited depth. So I keep my config table flat :3

Also I think you of course could reflect any kind of LUA table structure with JSON.

EDIT: Seems that the lib of course supports depth and I made a mistake. I will recheck that once again.

7

Actually the more I think of it, if the use case is a preset system then I don’t see the need to use JSON or XML when lua will do the job adequately.

I’ve been playing around with this lua table serialiser:http://lua-users.org/wiki/PickleTable which does the job just fine with a little extra code for file io

8

I have been looking at several scenarios to use external files to read or save data, to be able to read them later (read user preferences or save preferences of the tool):

1. ​** Save and read your own tool preferences (preferences.xml): The Renoise API seems to have its own XML interpreter?, so you can create a name.xml in the root of the installation folder of the tool (or subfolder) to save and then read your preferences. You can even use tables. To save the values in the table, it automatically create something like this<table_name_item>VALUE</table_name_item> for the name of each value, and you can access the name with table_name[NUMBER].value**.

--LUA----------------------
table_name = { VALUE_1, VALUE_2, VALUE_3, VALUE_4, VALUE_5 }

--XML----------------------
<table_name>
  <table_name_item>VALUE_1</table_name_item>
  <table_name_item>VALUE_2</table_name_item>
  <table_name_item>VALUE_3</table_name_item>
  <table_name_item>VALUE_4</table_name_item>
  <table_name_item>VALUE_5</table_name_item>
</table_name>

Tables are most useful for saving multiple values, for example, RGB color settings.The documentation is this (Renoise.ScriptingTool.API.lua):

--[[

Get or set an optional renoise.Document.DocumentNode object, which will be
used as set of persistent "options" or preferences for your tool.
By default nil. When set, the assigned document object will automatically be
loaded and saved by Renoise, to retain the tools state.
The preference XML file is saved/loaded within the tool bundle as
"com.example.your_tool.xrnx/preferences.xml".

A simple example:

    -- create a document
    my_options = renoise.Document.create("ScriptingToolPreferences") {
     some_option = true,
     some_value = "string_value"
    }

Or:

    -- create a document
    class "ExampleToolPreferences"(renoise.Document.DocumentNode)

      function ExampleToolPreferences:__init()
        renoise.Document.DocumentNode.__init(self)
        self:add_property("some_option", true)
        self:add_property("some_value", "string_value")
      end

      my_options = ExampleToolPreferences()

      -- values can be accessed (read, written) via
      my_options.some_option.value, my_options.some_value.value

      -- also notifiers can be added to listen to changes to the values
      -- done by you, or after new values got loaded or a view changed the value:
      my_options.some_option:add_notifier(function() end)

And assign it:

    -- 'my_options' will be loaded/saved automatically with the tool now:
    renoise.tool().preferences = my_options
]]

-- Please see Renoise.Document.API for more info about renoise.DocumentNode
-- and for info on Documents in general.
renoise.tool().preferences
  -> [renoise.Document.DocumentNode object or nil]

The first way seems simpler (it’s the one I use).Then, to save and read data, it is best to use XML following this method.

2. You want to read the data from other XML files that Renoise uses (read only, without modify). For example, the Config.xml (C:\Users\NAME_USER\AppData\Roaming\Renoise\V3.1.1\Config.xml).This file is on a "strange route"and, apparently, the tool will need an interpreter to read these files. I’m not sure.But it would be interesting to be able to access this data to adapt the tool if necessary. You never modify the user’s preferences, you only read the data.For example, access of the color of the button selection:

<SkinColors>
  ...
  ...
  <Selected_Button_Back>255,181,35</Selected_Button_Back>
  ...
  ...
</SkinColors>

How to read this data?Get print the table { 255,181,35 } ? In this case, it seems to be necessary to install an additional XML interpreter? Personally, it is a bit messy, just for wanting to access a few data.

**3.Use an LUA file (preferences.lua) to save data?**I wonder why is it necessary to use an XML file to save data (preferences.xml)?It is not possible to use a preferences.lua file, through require(“name”)? The preferences.lua should save the changed data when closing the tool.It would be the simplest thing, instead of using an XML file.Does not LUA save modified data in a file .lua?I think this would be less complicated to program.That is, it always uses the same programming language (LUA) for everything. Do not mix two languages.

9

**3.Use an LUA file (preferences.lua) to save data?**I wonder why is it necessary to use an XML file to save data (preferences.xml)?It is not possible to use a preferences.lua file, through require(“name”)? The preferences.lua should save the changed data when closing the tool.It would be the simplest thing, instead of using an XML file.Does not LUA save modified data in a file .lua?I think this would be less complicated to program.That is, it always uses the same programming language (LUA) for everything. Do not mix two languages.

I’ve been looking at this approach using the lua table serialiser I mentioned earlier. I can’t see any obvious advantage of XML over lua, but at the end of the day all of these parser’s whether XML, JSON or lua are going to generate strings that you will then need to write to a file, so the workload is the same regardless of which approach you take. It comes down to getting a library that will go both ways e.g. lua table to a string and vice versa…

10

For persistent data where loading or saving should be transparent, dealt with automatically (such as tool preferences), the Document API works great. For example, a tool like Duplex easily maintains a huge preferences XML file without a hitch.

But in the case of what OP is looking for, it won’t be of much use - you need to “model” the data structure in lua before you can import such XML data (and modelling Ableton Live project files sounds like quite the undertaking…)

Btw: I also wrote the lua string-serializer that I use in a number of other tools:

https://github.com/renoise/cLib/blob/master/classes/cLib.lua#L431

It comes down to getting a library that will go both ways e.g. lua table to a string and vice versa…

Indeed. I took this a step further and wrote a class (only used with AutoMate thus far) that not only takes a string and turns it into a lua table, but can also create actual luabind class instances (objects):https://github.com/renoise/cLib/blob/master/classes/cPersistence.lua.

It’s very specific to the way I’m working, as I rely heavily on lua-bind. But perhaps, some of the principles can be applied elsewhere?