Notes on scripting rtorrent

scripting-rtorrent.md

tested and source reviewed for rtorrent 0.9.6

d.multicall2=viewname,command performs command for each download that is visible in view viewname. This is not the same as each download with d.views.has=viewname. I think that d.views.* and view.* describe two entirely different things, or at best ones that are loosely related. Also, inconsistent naming and implicit targets just suck. A view (view.h) is a list of downloads split into visible downloads and filtered (not visible) downloads. set_visible(d) and set_not_visible(d) moves download d between the sublists, and filter_download(d). view.add=name -> core::control->viewManager.insert_throw(name) -> viewManager.insert(name) creates the new view and runs view.initialize(), which adds all existing torrents from the download list into the view (doing nothing to the downloads or d.views themselves)

When a download d is added (be it from session or new), view.set_visible=started or stopped is set based upon d.state

view.add=viewname:

• create new View view, initialize(viewname) -> get the control->core()->download_list and add each download in it to the view list, set m_size to the full size of the list and set m_focus to 0, then set last changed and add the "emit_changed_now" function to m_delayChanged, so each time emit_changed is called, emit_changed_now is added to the priority queue, which itself runs all the commands in m_signal_changed (currently appears only used for the hashing view and for displaying and updating views in the ui)
• add view to ViewManager list view.persistent=viewname:
• view->set_filter("d.views.has=viewname")
• view->set_event_added("d.views.push_back_unique=viewname")
• view->set_event_removed("d.views.remove=viewname")

how do quoted commands work? inner (escaped) quotes?

how do we push a new view onto a download properly? consider: "branch=(d.is_done),(( d.views.push_back_unique, done ))" vs "branch=(d.is_done),"d.views.push_back_unique=done"" (easily check with rtxmlrpc d.multicall2 '' main 'd.views=')

Looks like the predicate of d.multicall2 may have the same requirements as a line in a command file. Confirm.

Config files are loaded before session torrents are resumed

In later versions, will first look for rtorrent.rc in XDG_CONFIG_DIR/rtorrent/rtorrent.rc (if $XDG_CONFIG_DIR is defined and starts with /), then in $HOME/.config/rtorrent/rtorrent.rc (if $HOME is non-null), then in ~/.rtorrent.rc. In 0.9.6, only looks in ~/.rtorrent.rc.

Resuming torrents (i.e., adding them to the "main" view) does not happen before startup scripts run Does it work to delay the startup scripts a few seconds?

view.persist does not mean the view stays in the program after restart it stays on individual torrents, and d.views.has(viewname) is 1, and d.views lists viewname, but view.list does not list viewname, and d.multicall2=viewname,command doesn't work but rather fails with error "Input failed: Could not find view". Manually adding the view (view.add=viewname) adds viewname to view.list, d.views.has(viewname) is still 1, but d.multicall2=viewname,command runs command for all torrents, even those for which d.views.has(viewname) is 0 and don't have viewname listed in d.views. d.views.remove(viewname) does not change the d.views list, but does change d.views.has(viewname) to 0, and all torrents remain in the d.multicall2=viewname,command command list This stays the same with repeated attempts of d.views.remove(viewname) as well. This appears to be (at least partly) because of inconsistently adding the label vs a list containing only the label to the list of views; when d.views is a list, and an item of the list is a list whose only item is viewname, d.views.has=viewname is 0.

From source, it appears that when a view is created, all existing items on the download list are added to that view. (But not necessarily "visible") (ViewManager calls View::initialize() which pushes existing torrents onto the stack of views.) Probably means views need to be created in config files (i.e., before torrents are loaded) rather than manually. However, even though the torrents are added to the new view, the new view is not added to d.views. This is performed via command_download.cc: d.views.push_back => d_list_push_back(download,viewname,"rtorrent","views") => download_get_variable(download,"rtorrent","views").as_list().push_back(viewname) (with appropriate check if using d.views.push_back_unique). d.view.has=viewname checks over all views assigned to d (the download), rather than checking the downloads assigned to the view. d.multicall2 finds the view in view manager, then iterates from begin_visible() to end_visible();

Setting view to persistent (view.persistent=viewname) won't work if the view has already been modified. Persistence sets a filter on the view for d.views.has=viewname; an event for d.views.push_back_unique=viewname; and sets a remove event for d.views.remove=viewname.

Adding a download:

1. rtorrent/src/main.cc: main() => load_session_torrents() => new core::DownloadFactory(control->core()) for each session torrent => binds m_taskLoad.slot() to DownloadFactory::receive_load(); DownloadFactory::load(session torrent entry) => priority_queue_insert(scheduler, &m_taskLoad, time) => queue->push(m_taskLoad) => thread calls m_taskLoad.slot() => receive_loaded() => receive_success() => m_manager->download_list()->insert(download) => DownloadList::insert=>DL_TRIGGER_EVENT("event.download.inserted") (and others) => d.tracker.send_scrape, view.set_visible(started or stopped) => insert_visible(download), prioritize_toc
2. For new torrents, load.normal=filename => command_events.cc: apply_load(filename,create_tied|) => Manager::try_create_download_expand(filename, create_tied, commands) => try_create_download(filename,flags,commands) => create(filename,...) => DownloadFactory::load(filename) => then as above

main=>rpc::parse_command_single(,"try_import = <rtorrent.rc path>") => parse_command(target,string,end).first

rpc::parse_command

• [whitespace] = [\ \t]
• Each line must start with [whitespace]*[#[:isalpha:]] or you get "Invalid start of command name" => a line cannot only be "new" syntax
• individual commands (i.e., the command name, the first token) can only contain [\._[:isalnum:]] and must be < 128 char
• following the command must be [whitespace]*=
• then everything after = is put into torrent::Object args by rpc::parse_whole_list with delimiters [[isspace],;]
• skip the next [whitespace] if any
• if there's more and it starts with anything other than [\n;\0], give error "Junk at end of input."; point "first" to the end or to the character after those
• parse_command_execute(target,&args)
• return the result of commands.call_command(command,args,target) paired with the "first" pointer

rpc::parse_whole_list

• skip leading [whitespace]
• use rpc:parse_object to make a torrent::Object from the remaining string (a list, dict, quoted string, or string up to the first given delimiter)
• skip any following [whitespace]
• if the next character is ',', create a list and parse the rest of the string into it with parse_list

rpc::parse_object(first, last, dest, delim)

• if first char is {, parse text starting at the next character into torrent::Object::create_list() as dest with parse_list(first+1,last,dest,until parse_is_delim_block ([,}]) ), skip whitespace thereafter; if the next character after that isn't } (or you get to the end), error "Could not find closing '}'." Return pointer to the character after next.
• if first char is (, count & skip any following (; if there are more than 2 (i.e., more than 3 counting the first), error "Max 3 parantheses per object allowed." Keep track of the parenthetical "depth". Set dest as a torrent::Object::create_dict_key() and using flags to keep track of the depth (minus 1). Parse the following text into dest with parse_string(first+1,last,dest,is_delim_func(, or )) ). Skip any following [whitespace], then if reaching the end of input or something other than is_delim_func, error "Could not find closing ')'." If the next character is , then create a dictionary object that is a list, and parse text into it with parse_list(..., is_delim_func) and skip following whitespace. Finally, count closing parentheses and, if wrong, error "Parantheses mismatch."
• otherwise use parse_string to create string with the given delimiters

rpc::parse_list(first,last,dest,delim)

• skip [whitespace]
• parse text into temporary torrent::Object with parse_object and given delimiters
• skip [whitespace]
• push temporary object back onto list dest
• skip the next character and restart the loop until reaching the end or the next character is not ,

rpc::parse_string

• if empty (first == last), return without error
• if starts with " then push characters onto dest until reaching " (which then returns), end of input (which gives error "Missing closing quote."), or a \ at the end of input (which gives the error "Escape character at end of input.")
• otherwise, push characters onto dest until reaching a given delimiter, end of input, or \ at the end of input (which gives the error "Escape character at end of input.")

rpc::parse_command_execute(target,torrent::Object* args)

• if args is_list, take each arg that's not also a list, and run parse_command_execute(target,arg)
• else if args is_dict_key, do some more complicated parsing with flags and dict_obj
• otherwise, if args is a string starting with $, make args the result of parse_command(target, string without $)(without the "first" pointer)

CommandMap::call_command(key_type key, mapped type &arg, target_type target)

• find key in the command map; if not found, error "Command "<key>" does not exist."
• run the maplookup->second.m_anySlot(maplookup->second.m_variable, target, arg) and return the result

rpc::parse_command_multiple(target,first,last)

• calls parse_command(target,first,last) for each command separated by the delimiters in parse_command
• returns the (first part of the) result of the last command

Example, from command_download.cc initialize_command_download(): CMD2_FUNC_SINGLE("d.start", "d.hashing_failed.set=0 ;view.set_visible=started"); => CMD2_ANY("d.start", std::bind(&rpc::command_function_call_object, torrent::Object(torrent::raw_string::from_c_str("d.hashing_failed.set=0 ;view.set_visible=started")), std::placeholders::_1, std::placeholders::_2)); => CMD2_A_FUNCTION("d.start", command_base_callrpc::target_type, std::bind(&rpc::command_function_call_object, torrent::Object(torrent::raw_string::from_c_str("d.hashing_failed.set=0 ;view.set_visible=started")), std::placeholders::_1, std::placeholders::_2), "i:", "") => rpc::commands.insert_slot<rpc::command_base_is_typerpc::function::type>("d.start", std::bind(&rpc::command_function_call_object, torrent::Object(torrent::raw_string::from_c_str("d.hashing_failed.set=0 ;view.set_visible=started")), std::placeholders::_1, std::placeholders::_2), &rpc::command_base_callrpc::target_type, rpc::CommandMap::flag_dont_delete | rpc::CommandMap::flag_public_xmlrpc, NULL, NULL); => itr = rpc::commands.insert("d.start", rpc::CommandMap::flag_dont_delete | rpc::CommandMap::flag_public_xmlrpc, NULL, NULL); itr->second.m_variable.set_function(std::bind(&rpc::command_function_call_object, torrent::Object(torrent::raw_string::from_c_str("d.hashing_failed.set=0 ;view.set_visible=started")), std::placeholders::_1, std::placeholders::_2)); itr->second.m_anySlot = &rpc::command_base_callrpc::target_type;

Then d.start=target,arg (doesn't actually use arg) => rpc::command_base_callrpc::target_type(std::bind(&rpc::command_function_call_object, torrent::Object(torrent::raw_string::from_c_str("d.hashing_failed.set=0 ;view.set_visible=started")), std::placeholders::_1, std::placeholders::_2),target,arg) => check is_target_compatible, if not give error "Target of wrong type to command."; command_base::_call(std::bind(&rpc::command_function_call_object, torrent::Object(torrent::raw_string::from_c_str("d.hashing_failed.set=0 ;view.set_visible=started")), std::placeholders::_1, std::placeholders::_2),target,arg) => static_cast<command_base*>(std::bind(&rpc::command_function_call_object, torrent::Object(torrent::raw_string::from_c_str("d.hashing_failed.set=0 ;view.set_visible=started")), std::placeholders::_1, std::placeholders::_2))->_pod()(get_target_cast(target), arg); => rpc::command_function_call_object((raw_string as torrent::Object)"d.hashing_failed.set=0 ;view.set_visible=started",target,arg) => run d.hashing_failed.set=0 and view.set_visible=started, return result of view.set_visible=started

try_import=file => apply_try_import(file) => rpc::parse_command_file(file) and if unsuccessful log an error "Could not read resource file: " +

parse_command_file(file)

• open file stream
• while able to, read in a line of text, parse_count_escaped(of the line, which counts the number of \ at the end of the line); keep trap of line length, including multi-line commands with escaped newlines (though the escape characters \ at the end of the line themselves are not included in the count or the string); max line length is 4095 or you get error "Exceeded max line length." And if the number of escapes is odd, continue with the next line to the buffer (without running yet); if not, run parse_command(make_target(), buffer, length) and restart the loop.
• if a torrent::input_error occurs from one of the commands, loading stops and error is "Error in option file: filename:linenumber: error reason"
• otherwise, return true

Example: command that requires double-parenthesized expression (assume single line in file, terminated by newline):

• WRONG: branch=(d.is_dead),(d.process_dead)
  • parse_command_file => getline => char* buffer='branch=(d.is_dead),(d.process_dead)', gcount=length with newline, getcount=linelength=length without newline/terminator, no escapes at end => parse_command(make_target(),buffer,buffer+getcount) => parse_command_name(buffer,buffer+getcount,key,key+128) and parse_whole_list('(d.is_dead),(d.process_dead)')
    • => parse_object(&'(d.is_dead),(d.process_dead)', +length, dest, [[isspace],;] ) => parse_string(&'d.is_dead),(d.process_dead)',last,dest->as_dict_key,is_delim_func) => dest->as_dict_key='d.is_dead', first=&'),(d.process_dead)', then first=&',(d.process_dead)' then change dest to list (with first item 'd.is_dead'), parse_list(&'(d.process_dead)', length, dest, is_delim_command_func) => dest->as_list = args = { 'd.is_dead', dict_key('d.process_dead') }, first=last
  • parse_command_execute(make_target(), args) => parse_command_execute('d.is_dead') (as string and not starting with $, do nothing) and parse_command_execute(dict_key('d.process_dead')) (as dict key with empty object) => parse_command_execute(make_target(), dict_obj()) (which does nothing) then rpc::commands.call_command('d.process_dead', dict_obj(), make_target()) = immediately run d.process_dead

whereas this changes in the next-to-last step (parse_object) to set flags differently on the dict_key object. This is then interpreted by parse_command_execute as having different flags (specifically not flag_function), hence not immediately executed, and rather flags are reset to as though there were only one set of parentheses (so it will be interpreted next time it is sent through parse_command_execute).

Either way, this then goes through commands.call_command(key,args,target) and is returned as above