Was just playing around with Cas`s grouping tool that I just updated:
And managed to nest well beyond the renoise limit of 6 levels. You can see here in this screenshot, if you try and drag track 4 into group 8 then you get a renoise error:
To recreate this bug simply run the following code on a new empty song. You can obviously go from 1 to any number that you like, but 6 is already over the renoise limit and fires an error if you try and drag a track into the bottom group.
for i = 1, 6 do
renoise.song():insert_group_at(2)
renoise.song():add_track_to_group(1,2)
end