@taktik Thanks for the clarification - I’ll keep on dreamin’ 
2 Likes
We plan on including some native renoise granular and crossfading wavetable instruments in the next renoise release 
To be clear, these will be utilizing the current capabilities of renoise - so it’s stuff we can already do now - but will allow for some nice sonic possibilities for renoise users without having to reinvent the wheel and do all the setup work
5 Likes
Neat!
What’s up with that granular synthesis and other crap.
Can’t you put anything together anymore?
Something that would have a head and a heel? 
2 Likes
Just sent a pull request with the remaining definitions!
1 Like
The entire API is now converted and ready for use. Thanks a bunch to @unless for the help!
See remaining TODOs at GitHub - renoise/definitions: LuaCATS definitions for the Renoise Lua API for what’s still missing.
This really makes Lua editing a joy. Would be great if you guys could help testing all this to fix errors or missing docs.
6 Likes
Oh wow, thanks everyone! I hadn’t seen this so I was writing my own definitions as I needed them.
Is the readme status up to date? I will see if I can contribute.
sure, what are you looking for?
within the Renoise Scripting Editor? or do i need an external code editing app?
It won’t work with the built in script editor. You need to use a Language Server Protocol enabled editor and connect it to the Lua language server. The easiest to setup is Visual Studio Code with the LuaLS extension installed.
Edit: you can see how I use the definitions in my exs24 tool’s repo.
The .luarc.json tells LuaLS how to load the definitions.
The definitions are setup as a Git submodule like this:
mkdir -p definitions
git submodule add https://github.com/renoise/definitions.git ./definitions/renoise
I am looking into generating the docs.
I am able to spit out JSON and Markdown with this command:
mkdir -p build
lua-language-server --doc=./ --doc_out_path=./build --logpath=./build
To turn it into HTML, I am thinking we could use Pandoc. That would let us easily generate a PDF too.
Pandoc lets you write custom readers in Lua. This is executed by Pandoc directly and there are libraries available for JSON etc. So it’s pretty straighforward to build, and it’s Lua which should be familiar to contributors 
In addition to reading in the API docs, we can read in markdown, images, etc. to build the full docs site. So we can port over the documentation txt files and the images for the view builder API.
I would love to add a single page view, navigation, and (client side) search. It’s taking me forever to find things with the current docs. And if we are using Pandoc I would like to spit out Dash / Zeal docs too 
If someone could look into the document generation this indeed would be great and help a lot. Thanks for considering that @mattya.
The problem with the JSON from the Lua language server seems to be that it contains far too much information in a rather unhelpfully structured way for the API docs, including all the standard Lua API symbols and so on. I am not sure how easy it would be to exclude these and structure the resulting PDF in a custom way, similar to the old docs:
https://files.renoise.com/xrnx/documentation/
How would you do the JSON processing, API doc filtering & sorting?
Like the Pandoc idea though. I think @unless has tested/considered using using mdBook here as well.
Hey, I already started restructuring and extending the txts from the manuals into markdown files. Currently using mdbook (it has a fast search built-in as well) to generate a html view but once everything is in markdown any other doc gen could work.
As for the definitions reference, it would be great to have some readable format! I got to excluding all the irrelevant parts from the JSON output but it’s still rather disconnected in its structure so some custom logic has to be written to parse everything into meaningful chunks of classes and such.
1 Like
Here is a shell script that you can call with a lua definition file and it will output a .json with the same name with any standard lua stuff removed.
#!/bin/bash
lua-language-server --doc="$1" --doc_out_path="."
jq --arg search $1 '[.[] | select(any(.defines[]; .file | contains($search)))]' doc.json > "$1.json"
rm doc.json doc.md
jq '.[] | .name' "$1.json"
echo "-> $1.json"
Essentially all renoise entries will contain the source file’s path while the standard stuff will not.
Nice, appreciate all the input!
The JSON output is unfortunate. I’m using a similar trick to @unless ATM where I filter out types that aren’t defined in the current working directory. Some of them would go away if we changed how the definitions are written (e.g. the class API has a function definition, which causes the functions it calls to be included), but others I can’t get rid of.
The structure is more problematic. We can get a pretty decent structure just by splitting on the namespace dots in the type name and inferring the hierarchy. But that won’t match the old docs, because for example renoise.Sample won’t be nested under renoise.Song. It might work better to use the file structure. That is what rust doc does. That still doesn’t match the old docs but it’s close: Song > Instrument > Sample.
The JSON processing, filtering, and sorting can all be done in the Pandoc Lua reader. I have the basics working - I’m able to document a single function signature.
That’s great you already started converting the txt files. I think we could go either way: put the API docs in mdbook or put merge the markdown files into pandoc. With the pandoc stuff the same code can render to markdown or HTML. So it could generate markdown that then gets inserted into mdbook.
Using mdbook would actually solve the structure issues, since we could manually organize the “entry points” to the API docs from there. This isn’t using mdbook but the Elixir ecosystem does a good job of combining guides and API docs in a similar format: Ecto — Ecto v3.11.2
Edit: This is how it would look if we merged the default Markdown output into mdbook:
With mdbook we can probably skip pandoc completely. A Rust preprocessor could parse the JSON and generate the markdown on it’s own.
2 Likes
I like the mdbook stuff. Especially if that means that we can merge the general, custom API descriptions (the old txt contents from here) and the API docs together.
Perhaps the file/folder structure of definitions/library at main · renoise/definitions · GitHub can be used as a template, programmatically for the API doc structure? Then the structure doesn’t need to be hardcoded.
2 Likes
Heh, I came to the conclusion that using the file structure made more sense after trying the namespacing approach. That’s roughly how rustdoc works too.
However with mdbook (at least by default) you need to list the pages in SUMMARY.md. So at least the “entrypoints” for the API would need to go there. For the example above I had this:
# Summary
[Introduction](./introduction.md)
# User Guide
- [Enable scripting](./enable-scripting.md)
# API Reference
- [Standard library](./standard.md)
However, it seems like a preprocessor could be used to inject the API docs. So we could have a directive like this:
{{#luadoc definitions/renoise.lua}}
…that would expand to the chapter list, and inject the pages.
The simplest thing to do by far is to only have the preprocessor build the definition markdown files, and write the links in SUMMARY.md manually. Then for each API reference we can optionally write prose and include the generated markdown like this:
# Some API
Some optional prose
{{#include definitions/somedefinition.lua}}
Edit: it is possible to add pages with a preprocessor, there’s an implementation here: mdbook-all-the-markdowns — Rust utility // Lib.rs
I started a mdbook preprocessor here: GitHub - matt-allan/mdbook-luacats: A mdbook preprocessor for LuaCATS API documentation
The mdbook part is just a stub ATM. The core logic is being built out separately, with a test binary that simply prints rendered markdown to stdout.
This is the output it currently generates for the bit.lua file: bit.md · GitHub
I will try and get this finished but I might not before I am unavailable for a month or two, so feel free to fork this if I am not responding.
2 Likes
Well I had more free time than anticipated and I spent a very long time trying to parse the LuaCats doc.json. I ended up making a new repository because I had to start from scratch: GitHub - matt-allan/mooncats: Mdbook support for LuaCats API docs.
It sort of works but the JSON docs are really not good. I don’t think it’s worth the effort. Parsing the definition files ourselves (similar to how the existing docs are built) is probably a better idea.
The doc representation is clearly optimized for IDE usage. Even after doing all the work to clean up the data so it can be used it’s incomplete. There is data we need that just isn’t there.
A few examples of the issues:
- Enum fields don’t specify the enum value or even type
- Types don’t specify optionality (e.g.
integer?) - Fields like
renoise.API_VERSION = 6.1just show up asnumber - The “view” (source code) for type aliases is truncated if it’s too long
I did get to see how it would look though. Here’s a screenshot:
(We could add custom CSS, that’s just the default styling for generated markdown)
Mirroring the directory structure seems to work well.
If I try this again I will probably try to write the preprocessor in Lua and use LPeg to parse the definitions. Not sure if I can sink any more time into this ATM.