Copy main directory & your subdirectories & files with Lua

Renoise Tool Development
1

Hi. I’m trying to create a function that does the following (clone a directory with all its content):

The stage:

  • I have a main_folder folder. It is a fixed folder.
  • Inside main_folder I have several subfolders numbered, with names numbered like this: 01, 02, 03 … 99 (up to 99 subfolders).
  • Inside main_folder I have a main.xml file.
  • Within each subfolder, I have several sub_files.xml files.

Build the function:
How to build the os_copy() function with LUA.
LUA does not seem to have an option like os.copy(“origin”,“destiny”) to work.

What would be the most correct way to make a backup of main_folder and all its contents (subfolders with more xml files)?

The idea is to press a button, a pop-up window will appear to define the destination path, and there clone the main_folder folder and all its contents.

The content is very simple, they are only folders and xml files with certain order (it is possible to iterate). The most unfavorable case would be to copy about 5MB of content. That is, it is not a large amount of data.

Any ideas to address this and work on MAC, Linux and Windows? I think this will not be complicated. I am more interested in how to build the cloning function of a complete directory and its content…

2

I’ve done something similar. It should be a recursive function.

Let me know if you need me to make a bytecompiled function and what fee you are willing to pay.

1 Like
3

rprint(io)
[chmod] => function: 0000000017B7E820
[close] => function: 00000000195B1EF0
[exists] => function: 0000000017B7E700
[flush] => function: 00000000195B1BF0
[input] => function: 00000000195B1F20
[lines] => function: 00000000195B1A40
[open] => function: 0000000017B7E8B0
[output] => function: 00000000195B1DD0
[popen] => function: 00000000195B1B30
[read] => function: 00000000195B1BC0
[stat] => function: 0000000017B7E850
[stderr] => file (0000000141673CE0)
[stdin] => file (0000000141673C30)
[stdout] => file (0000000141673C88)
[tmpfile] => function: 00000000195B1C50
[type] => function: 00000000195A11E0
[write] => function: 00000000195A1600

rprint(os)
[clock] => function: 0000000017B7EAC0
[currentdir] => function: 0000000017B7E8E0
[date] => function: 00000000195A1B40
[difftime] => function: 00000000195A1540
[dirnames] => function: 0000000017B7EBE0
[execute] => function: 0000000017B7ECD0
[exit] => function: 0000000017B7E460
[filenames] => function: 0000000017B7E430
[getenv] => function: 00000000195A19F0
[mkdir] => function: 0000000017B7EC70
[move] => function: 0000000017B7EB80
[platform] => function: 0000000017B7E6D0
[remove] => function: 0000000017B7E7C0
[rename] => function: 0000000017B7E670
[setlocale] => function: 00000000195A1240
[time] => function: 00000000195A12A0
[tmpname] => function: 0000000017B7E640

Inside I os, I think it’s possible to do it with os.execute. I just need a little guidance. I think I could build the function myself.

4

The recursive filelist method would look similar like:

local getRecursiveFileList = function (path)
  local result = {}
  local c = 1
  local fileList = dir(path)
  for x, filepath in ipairs(fileList) do
    if (filepath == "..") then
      continue
    end
    if (typeof(filepath) == FILESYSTEM.DIR) then
      result = array_concat(getRecursiveFileList(filepath), result)
    else
      result[c] = filepath
      c = c + 1
    end
  end
 return result
end

Then you would loop over the filelist result, and copy each file from the original path to your new dir + filepath with stripped away old dir.

The actual OS call is something like:

os.filenames(path) to get only the files of a dir.
os.dirnames(path) to get only the subdirs.

So you would adjust the above function by first looping over files and then looping over subdirs separately, each self calling the function and concatting the result with your current loop result (the actual recursion).

Note: This info was released under GPL license and may not be used for copyprotections

5

@ffx, thanks!
From what I see, basically it is to check that both the directories and the files exist, and if they exist, copy them from the original route to the new route (without deleting anything).

As my case is very specific, I just need to access two levels of directories (one main folder and the rest are subfolders of the same level), with a function that reaches that level is sufficient.

I think I already have a clear idea of the concept. But what I’m thinking does not reach all possible sub-folder levels. Perhaps in this case it is convenient not to go into such depth.

Are these “recursive functions” for all files and directories of all sublevels?

I have not experienced much on this subject. So I do not know to what extent it can be reasonable (how many MB can be copied with these methods without problems).

6

You can add a level/depth value to the function, like:

local getRecursiveFileList = function (path, level)
  ...
  level = level + 1
  if (level <= 2) then
    ...getRecursiveFileList(filepath, level)...
  end
  ...
end

And then you call the function with level = 0.

7

I just wanted information to see how to approach it. The 99.9% of the code that I do for my tools is written by me (inside of that can be considered our own code, since we all use LUA). I’m interested in learning to do it.

In this particular case, it is not very clear how LUA faces the cloning of files. In fact, I’ve been surprised by this issue. Apparently, there is no os.copy (origin, destination) or something similar, and I’m not the first to look for this.

@joule @ffx thanks for the help!

Finally, I opted for a non-recursive function. Only access to 2 levels of subdirectories. It is a very concrete way to make a backup of small files through a button. It can probably be improved. There must be some more direct (and quicker) way to copy one file to another without having to open and close it.

The function that I have done is like this:

--backup all banks
function backup_all_banks()
  --check if exist main_folder
  local origin_main_folder="chord_banks"
  if io.exists(origin_main_folder) then
    --create destiny main folder
    local path=rna:prompt_for_path("Choose a destination folder to save a backup of all the banks.")
    local destin_main_folder=("%s/%s"):format(path,origin_main_folder)
    --check if exist destination folder
    if io.exists(destin_main_folder) then
      local mpt=rna:show_prompt((" %s"):format(main_title),("The folder %s already exist.\nDo you want to overwrite it?"):format(destin_main_folder),{"Ok","Cancel"})
      if (mpt=="Ok") then
      else
        return
      end
    end
    --local x=os.clock()
    os.mkdir(destin_main_folder)
    local mainfiles=os.filenames(origin_main_folder)
    --rprint(mainfiles)
    
    --copy main files
    for main_fls=1,#mainfiles do
      local orig=("%s/%s"):format(origin_main_folder,mainfiles[main_fls])
      local dest=("%s/%s"):format(destin_main_folder,mainfiles[main_fls])
      --open and read content file
      local old_file=io.open(orig,"r")
      local content=old_file:read("*a")
      old_file:close()
      --copy the content
      local new_file=io.open(dest,"w")
      new_file:write(content)
      new_file:close()
    end
    
    --copy subfolders
    local subfolders=os.dirnames(origin_main_folder)
    --rprint(subfolders)
    for sub_fld=1,#subfolders do
      if io.exists(("%s/%s"):format(origin_main_folder,subfolders[sub_fld])) then
        --create subfolders 
        os.mkdir(("%s/%s"):format(destin_main_folder,subfolders[sub_fld]))
        --check files into subfolders
        local subfiles=os.filenames(("%s/%s"):format(origin_main_folder,subfolders[sub_fld]))
        --rprint(subfiles)
        --copy files into subfolders
        for sub_fls=1,#subfiles do
          local orig=("%s/%s/%s"):format(origin_main_folder,subfolders[sub_fld],subfiles[sub_fls])
          local dest=("%s/%s/%s"):format(destin_main_folder,subfolders[sub_fld],subfiles[sub_fls])
          --open and read content file
          local old_file=io.open(orig,"r")
          local content=old_file:read("*a")
          old_file:close()
          --copy the content
          local new_file=io.open(dest,"w")
          new_file:write(content)
          new_file:close()
        end
      end
    end
    --print(string.format("folder cloned: %.2f ms\n",(os.clock()- x)*1000))
  else
    local mpt_2=rna:show_prompt(
      (" %s"):format(pre_main_title),("The folder \"%s\" not exist! Operation cancelled.\n"..
      "Check that it is inside the root folder of the %s tool."):format(origin_main_folder,main_title),{"Close"}
    )
  end
end
8

I have been able to advance a little more here. With the previous function as a base, it is necessary to use coroutines (coroutine.yield()) through the last iteration so that it works with thousands of small files without the pop-up window appearing every 10 seconds. I’ve run tests with a copy of over 25,000 small files (around 1KB) and it lasts about 20 seconds or more with my computer. Apparently, copying a single heavy file is not the same as thousands of small files, since you have to write thousands of routes instead of just one.

If anyone has any idea how to improve this, I would greatly appreciate it.

As I researched on Google, there are several ways to copy a file from one folder to another with LUA. Do you know what is the fastest way to do this concrete step (copy only a file)?