The output of testScript.sh shows that chatbot_civicrm_alterSettingsFolders is being called but for some reason, the folder is not always added to $metaDataFolders.
Next step is to dig into \Civi::dispatcher()->dispatch('hook_' . $fnSuffix, $event);
I have had a look at Civi\Core\Event\GenericHookEvent but am struggling to work outwhere the dispatch method is (would be nice to step through with a debugger but do not have one configured).