-
-
Save tavinus/93bdbc051728748787dc22a58dfe58d8 to your computer and use it in GitHub Desktop.
#!/usr/bin/env bash | |
############################################################ | |
# MIGRATED TO REPOSITORY | |
# https://github.com/tavinus/cloudsend.sh | |
# | |
# This gist will NOT be updated anymore | |
############################################################ | |
############################################################ | |
# | |
# cloudsend.sh | |
# | |
# Uses curl to send files to a shared | |
# Nextcloud/Owncloud folder | |
# | |
# Usage: ./cloudsend.sh <file> <folderLink> | |
# Help: ./cloudsend.sh -h | |
# | |
# Gustavo Arnosti Neves | |
# https://github.com/tavinus | |
# | |
# Contributors: | |
# @MG2R @gessel | |
# | |
# Get this script to current folder with: | |
# curl -O 'https://raw.githubusercontent.com/tavinus/cloudsend.sh/master/cloudsend.sh' && chmod +x cloudsend.sh | |
# | |
############################################################ | |
CS_VERSION="0.1.6" | |
CLOUDURL="" | |
FOLDERTOKEN="" | |
PUBSUFFIX="public.php/webdav" | |
HEADER='X-Requested-With: XMLHttpRequest' | |
INSECURE='' | |
# https://cloud.mydomain.net/s/fLDzToZF4MLvG28 | |
# curl -k -T myFile.ext -u "fLDzToZF4MLvG28:" -H 'X-Requested-With: XMLHttpRequest' https://cloud.mydomain.net/public.php/webdav/myFile.ext | |
log() { | |
[ "$VERBOSE" == " -s" ] || printf "%s\n" "$1" | |
} | |
printVersion() { | |
printf "%s\n" "CloudSender v$CS_VERSION" | |
} | |
initError() { | |
printVersion >&2 | |
printf "%s\n" "Init Error! $1" >&2 | |
printf "%s\n" "Try: $0 --help" >&2 | |
exit 1 | |
} | |
usage() { | |
printVersion | |
printf "\n%s%s\n" "Parameters:" " | |
-h | --help Print this help and exits | |
-q | --quiet Be quiet | |
-V | --version Prints version and exits | |
-k | --insecure Uses curl with -k option (https insecure) | |
-p | --password Uses env var \$CLOUDSEND_PASSWORD as share password | |
You can 'export CLOUDSEND_PASSWORD' at your system, or set it at the call. | |
Please remeber to also call -p to use the password set." | |
printf "\n%s\n%s\n%s\n" "Use:" " $0 <filepath> <folderLink>" " CLOUDSEND_PASSWORD='MySecretPass' $0 -p <filepath> <folderLink>" | |
printf "\n%s\n%s\n%s\n" "Example:" " $0 './myfile.txt' 'https://cloud.mydomain.net/s/fLDzToZF4MLvG28'" " CLOUDSEND_PASSWORD='MySecretPass' $0 -p './myfile.txt' 'https://cloud.mydomain.net/s/fLDzToZF4MLvG28'" | |
} | |
########################## | |
# Process parameters | |
if [ "$1" = "-h" ] || [ "$1" = "--help" ]; then | |
usage | |
exit 0 | |
fi | |
if [ "$1" = "-V" ] || [ "$1" = "--version" ]; then | |
printVersion | |
exit 0 | |
fi | |
if [ "$1" = "-q" ] || [ "$1" = "--quiet" ]; then | |
VERBOSE=" -s" | |
shift | |
fi | |
if [ "$1" = "-k" ] || [ "$1" = "--insecure" ]; then | |
INSECURE=' -k' | |
log " > Insecure mode ON" | |
shift | |
fi | |
if [ "$1" = "-p" ] || [ "$1" = "--password" ]; then | |
PASSWORD=${CLOUDSEND_PASSWORD} | |
log " > Using password from env" | |
shift | |
fi | |
########################## | |
# Validate input | |
FILENAME="$1" | |
CLOUDURL='' | |
# if we have index.php in the URL, process accordingly | |
if [[ $2 == *"index.php"* ]]; then | |
CLOUDURL="${2%/index.php/s/*}" | |
else | |
CLOUDURL="${2%/s/*}" | |
fi | |
FOLDERTOKEN="${2##*/s/}" | |
if [ ! -f "$FILENAME" ]; then | |
initError "Invalid input file: $FILENAME" | |
fi | |
if [ -z "$CLOUDURL" ]; then | |
initError "Empty URL! Nowhere to send..." | |
fi | |
if [ -z "$FOLDERTOKEN" ]; then | |
initError "Empty Folder Token! Nowhere to send..." | |
fi | |
########################## | |
# Check for curl | |
CURLBIN='/usr/bin/curl' | |
if [ ! -x "$CURLBIN" ]; then | |
CURLBIN="$(which curl 2>/dev/null)" | |
if [ ! -x "$CURLBIN" ]; then | |
initError "No curl found on system!" | |
fi | |
fi | |
########################## | |
# Extract base filename | |
BFILENAME=$(/usr/bin/basename $FILENAME) | |
########################## | |
# Send file | |
#echo "$CURLBIN"$INSECURE$VERBOSE -T "$FILENAME" -u "$FOLDERTOKEN":"$PASSWORD" -H "$HEADER" "$CLOUDURL/$PUBSUFFIX/$BFILENAME" | |
"$CURLBIN"$INSECURE$VERBOSE -T "$FILENAME" -u "$FOLDERTOKEN":"$PASSWORD" -H "$HEADER" "$CLOUDURL/$PUBSUFFIX/$BFILENAME" |
whats this "https://cloud.mydomain.net/s/fLDzToZF4MLvG28" whats a "folderlink" where do i get it from? specially the "s/fLDzToZF4MLvG28" part
whats this "https://cloud.mydomain.net/s/fLDzToZF4MLvG28" whats a "folderlink" where do i get it from? specially the "s/fLDzToZF4MLvG28" part
You have to share a Folder writable and use the generated link
Any way to make this use a password protected share?
is it working with nc15? how to debug sending problems?
I've created a fork to add a more quiet mode and to allow working with passwords. Pretty new to actually working with gists... I'm assuming there's no PRs here?
Hi there! Sorry for the delay. This gives no notifications of messages. Also, no PRs on gists ...
I like your changes @MG2R , I will try to add them here.
I guess I will import your fork to a full repo (next week).
Just need to add some parameter parsing so we can pass them in any order (that is easy, I have it ready from other scripts).
Apart from this, I have another script called cloudmanager
that can do pretty much anything that webdav allows.
But it is in Brazilian Portuguese only for now. I will see about making it multilanguage or translating it.
I will post here when I do that.
Here is the help in Portuguese (the commands are in English, so you can guess it):
$ cloudmanager --help
Arnosti Cloud Manager v0.0.5
OPCOES
-V, --version Imprime versao na tela e termina execucao
-h, --help Imprime esta ajuda e termina execucao
COMANDOS
> ls,list <destino>
Lista uma pasta ou arquivo
Ex: cloudmanager list
> mkdir,makeFolder <nomedapasta>
Cria uma pasta no cloud
Ex: cloudmanager mkdir PastaTeste
> send,upload <pasta/arquivo.local> <pasta/arquivo.cloud>
Envia um arquivo local para o cloud
Ex: cloudmanager send '/user/nfe/meuarquivo.xml' 'ARCO/nfe/meuarquivo.xml'
> mv,move <origem> <destino>
Move um arquivo ou pasta dentro do cloud
Ex: cloudmanager move 'ARCO/meuarquivo.txt' 'ATL/meuarquivo.txt'
> cp,copy <origem> <destino>
Copia um arquivo ou pasta dentro do cloud
Ex: cloudmanager copy 'ARCO/meuarquivo.txt' 'ATL/meuarquivo.txt'
> del,delete <destino>
Apaga um arquivo ou pasta dentro do cloud (move para a lixeira)
Ex: cloudmanager del 'ARCO/meuarquivo.txt'
> get,download <origem> <destino>
Faz download de um arquivo do cloud para um disco local
Ex: cloudmanager get 'ARCO/meuarquivo.txt' '/user/nfe/meuarquivo.txt'
> shares,listShares
Imprime tabela com compartilhamentos (183 cols)
Especifique uma pasta ou deixe em branco para listar tudo.
Ex: cloudmanager shares
> links,sharedLinks
Lista os compartilhamentos por links em formato <URL>,<NOME ARQUIVO>
Ex: cloudmanager listShares
This one needs a config file for authentication though (for obvious reasons).
Cheers!
Hi,
any inclination to release your cloudmanager script ?
The following one-liner downloads all files from a password-protected shared folder :
(xml_grep
dependency ; could also be done with xmllint
or xpath
)
SRV={full server URL}
USR={folder token}
PW={password}
for url in $(curl -s -X PROPFIND -u ${USR}:${PW} ${SRV}/public.php/webdav/ | xml_grep --text_only "//d:multistatus/d:response/d:href" | egrep -v '/$') ; do echo "Downloading : $url ..." ; curl -sJO -X GET -u ${USR}:${PW} ${SRV}/$url ; done
I've been so busy mate..
x_x
Any way to make this use a password protected share?
I'm not sure if you discovered this already, but the :
in the -u
flag is a separator between the username and password.
i.e.
-u "$FOLDERTOKEN":"MyPassword"
is it working with nc15? how to debug sending problems?
Anyone can confirm? I am testing with nc17, there is no error after the upload, but no file appears in the cloud...
Hi all.
I have updated the app with changes from others and mine.
v0.1.5
Just tested sending files to a Nextcloud 18.0.1 instance share with and without password and both worked fine.
Shares with passwords
To upload to a share with password set you need to
- Set environment variable
$CLOUDSEND_PASSWORD
AND - Use the
-p | --password
flag on call
To set the env var password
To set the variable you can either do it system-wide (not too secure)
export CLOUDSEND_PASSWORD='MySecretPass'
Or you can define at the time of the call, appending as in the --help
examples.
There are many ways to implement it this way, but the idea is that the password var is defined only in the context of the bash instance that it is running.
Also, be aware that the password will probably end up at your bash history file if you call it from a terminal, but will not show up on the process listings, like on ps, top, htop.
Help info updated
I have appended the password information to the --help
info and also added examples on how to set password at the call time.
$ ./cloudsend.sh --help
CloudSender v0.1.5
Parameters:
-h | --help Print this help and exits
-q | --quiet Be quiet
-V | --version Prints version and exits
-k | --insecure Uses curl with -k option (https insecure)
-p | --password Uses env var $CLOUDSEND_PASSWORD as share password
You can 'export CLOUDSEND_PASSWORD' at your system, or set it at the call.
Please remeber to also call -p to use the password set.
Use:
./cloudsend.sh <filepath> <folderLink>
CLOUDSEND_PASSWORD='MySecretPass' ./cloudsend.sh -p <filepath> <folderLink>
Example:
./cloudsend.sh './myfile.txt' 'https://cloud.mydomain.net/s/fLDzToZF4MLvG28'
CLOUDSEND_PASSWORD='MySecretPass' ./cloudsend.sh -p './myfile.txt' 'https://cloud.mydomain.net/s/fLDzToZF4MLvG28'
Thanks @MG2R for the password implementation and to @gessel for the late fixes.
Cheers! 🍺
Gus
Hi,
any inclination to release your cloudmanager script ?
Hi!
I have rebuilt my CloudManager App and released the code as public AGPLv3.
https://github.com/tavinus/cloudmanager
Please note that the CloudManager App is NOT a replacement for this CloudSend script.
The tools are actually complimentary.
- CloudManager is a fully capable webdav client; able to access Nextcloud using a Nextcloud account / password
- CloudSend is a simple client designed to send files to public shared folders using the shared folder link / password
I will probably eventually include CloudSend into CloudManager, or else I will end up migrating this gist to a full repository.
CloudManager Features
(as of 2020/03/16)
Tries to be feature-complete with the Nextcloud Webdav's capabilities.
Webservice Tasks
- list file/folder
- create folder
- send file
- get file
- delete file/folder
- move file/folder
- copy file/folder
- list shares
- list links
Internal App Goals
- target folder detection redirection (ends with /)
- external config files
- reset config files
- multiple users
- multi-language support
- auto-detect / auto-load system language
- force language from param
Ola Gustavo, https://gist.github.com/tavinus/93bdbc051728748787dc22a58dfe58d8#file-cloudsend-sh-L103 breaks up nc 18.0.2.
Reverting to L102, does the trick:
With L103: /usr/bin/curl -T testfile.txt -u 3RZB8ywFcLAaYX8: -H X-Requested-With: XMLHttpRequest https://myserver/s/3RZB8ywFcLAaYX8/public.php/webdav/testfile.txt <- no good
With L102: /usr/bin/curl -T testfile.txt -u 3RZB8ywFcLAaYX8: -H X-Requested-With: XMLHttpRequest https://myserver/public.php/webdav/testfile.txt <- good call
Hi @michimau,
Seems like the problem is not really with the version of Nextcloud, but with how it generates the shared links.
There seems to be 2 different possibilities on how these links are created. I tested on 2 different instances, one running NC18 and another NC16. Funny thing is that the NC16 is the one that created links equal to yours, while the NC18 created different links.
The two possibilities
So one has /index.php/
and the other one does not.
The ones with index.php
require L103 while the others require L102.
Having nextcloud running in a folder seems to be ok in either case (eg. domain.tld/nextcloud)
I am not sure why this is not consistent (why some times it has index.php and some times not).
In any case, I have added a check for the index.php
part in the URL and then it will be processed either with what was Line 102 or with what was in Line 103 automagically.
Hopefully this will make it work for everyone and in every version.
I have updated the script to the new version with the fixes ( v0.1.6 ).
Edit: The only thing I can think about is that one server is running on nginx and the other one on Apache (even though it is routed through an nginx proxy as well). So maybe the difference in the link comes from some apache .htaccess or something.
Cheers ☕
Gus
Thank you for the patch, simple and effective.
I have tested the following setups in docker environments:
haproxy -> nc-apache
haproxy -> nginx -> nc-fpm
and the behavior has been consistent within 18.0.x versions.
I see... who knows, maybe there is a config for that and we are missing it.
Thanks for reporting back.
Hi @tavinus
any way to to create a folder named "my Sub Folder" (filenames with blanks) ?
pi@baumgarten:~/cloudmanager $ ./cloudmanager.sh makeFolder 'my sub folder'
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
<hr>
<address>Apache/2.4.38 (Debian) Server at 172.18.0.3 Port 80</address>
</body></html>
Hi.
I will answer it here for completeness, but please let us continue this on the cloudmanager issue.
This has actually always been possible, but there is no recursive folder creation.
The parent folder must exist. So you need one call for each folder level to be created.
It should work for files and folders with spaces on both source and target locations.
Names must be quoted or escaped, eg:
"my file name"
my\ file\ name
Thanks for letting me know of this issue.
Please let me know if this works for you.
Cheers! 🍺
Gus
I would like to alias or otherwise hide the URL to the shared folder away from the command, so I could simply execute cloudsend file
.
Do you think it'd be possible to put the URL behind a flag (-u --url ?) instead of a positional argument so that I could do this in a clean way?
Hi, there are many ways to do that.
You could create a separate script that calls cloudsend.sh $1 <hardcodedUrl>
.
So <file>
would come from $1
(first parameter) and the URL could be a fixed variable or the URL string.
Example (untested)
#!/bin/bash
CLOUDSENDBIN='/path/to/cloudsend.sh' # Using full path can avoid errors
SHAREURL='https://cloud.domain.tld/s/xdFvws' # Your folder share URL
"$CLOUSENDBIN" "$1" "$SHAREURL"
If the share has a password that should also be added.
Example (untested)
#!/bin/bash
CLOUDSENDBIN='/path/to/cloudsend.sh' # Using full path can avoid errors
SHAREURL='https://cloud.domain.tld/s/xdFvws' # Your folder share URL
CLOUDSEND_PASSWORD='MySecretPass' "$CLOUSENDBIN" -p "$1" "$SHAREURL"
Or you could modify this gist and hard code the URL inside it as well (around line 103).
https://gist.github.com/tavinus/93bdbc051728748787dc22a58dfe58d8#file-cloudsend-sh-L103
Will have to add a new var and use it in place of $2
.
Example mod (untested)
SHAREURL='https://cloud.domain.tld/s/xdFvws' # Your folder share URL
CLOUDURL=''
# if we have index.php in the URL, process accordingly
if [[ $SHAREURL == *"index.php"* ]]; then
CLOUDURL="${SHAREURL%/index.php/s/*}"
else
CLOUDURL="${SHAREURL%/s/*}"
fi
FOLDERTOKEN="${SHAREURL##*/s/}"
I would argue that it is cleaner the way it is now though.
It behaves like copy A B
, which makes more sense for me than using a --url
parameter.
So I don't think I will change the main script to behave in a different way.
Cheers! 🍺
Thanks for the lengthy response with examples! I'd prefer to avoid hardcoding modifications or creating secondary scripts though.
I think there's a way that'd satisfy both use cases - how about adding support for both positional parameters and flags?
The script would still be convenient to use by real people, but you could use it in scripts/aliases without hardcoding modifications or ugly workarounds.
I actually went ahead and did just that(along with some other small changes), please take a look - https://haste.rys.pw/raw/afisuwosoz
You can either use the positional arguments like until now, or you can use --file
together with --url
(have to use both or nothing) and positioning is then ignored.
All the code is there, what's left is to modify usage() to reflect the new changes.
Hi again.
I guess that if you already have a fork that works the way you want, you are good to go.
Like I said, I see no reason to add that complexity and I prefer it the way it is ( copy <source> <destination>
).
Cheers!
I'm not sure if we're on the same page - copy <source> <destination>
is still available with my changes, my point was to not break existing usage, it's just that you can ALSO use parameters for easy use in scripts.
There's no real 'complexity' added, I simply put the Process parameters
stage into a case statement (which is cleaner) and added a condition if it encounters an unknown(positional) parameter.
I'd prefer to have my changes merged rather than have to maintain my own fork.
I've updated the URL to a file with the usage() changes already present.
@tavinus You may want to move everything here into a github repo, it's maybe getting too large for a single gist ^^
Migrated to repository
https://github.com/tavinus/cloudsend.sh
This gist will not be updated anymore.
Rebuild
I have just rebuilt most of the script and added many improvements.
But it is only available at the new repository.
This gist will be kept as is for all eternity.
The only 'downside' is that the old -p
option is now -e
, because -p
now receives the password as a parameter, as in -p <myPass>
.
From here to there, this is the current log (Aug 2020)
- v0.1.7 - Rename
- Adds option
-r
/--rename
to change the destination (uploaded) file name.
- Adds option
- v2.0.0 - Rebuild
- Script rebuild; many internal changes
- Changes how options and parameters are parsed
- Added many validation and checking functions
- Options can now come in any order and after the filename and URL Link
- Updated and added verbose messages
- Updated quiet mode
- Updated
--help
andreadme.md
- Password parsing has changed and is NOT compatible with prior versions
-p
now asks for a password as a parameter, as in-p <pass>
- new option
-e
uses an envrionment variable password (substitutes the old-p
option)
- v2.1.0 - Pipes and Globs
- Adds stdin as input option (piped content)
- uses either
-
or.
as input file name - requires
-r <filename>
- uses either
- Allows to use globbing at the input
- can send multiple files by using globs
- requires
-g
option - not compatible with
-r <name>
option (rename)
- Adds new checks and parsing
- Cleaned up unused code
- Updated readme and
--help
- Adds stdin as input option (piped content)
- v2.1.1 - Pipes and Globs (fix)
- Small fix to file name parsing VS globbing
- Updated readme globbing link
- v2.1.2 - Pipes and Globs (readme-help)
- Better help and readme text and examples.
- v2.1.3 - Pipes and Globs (fix2)
- Disables curl globbing when not globbing.
More info on globbing:
- https://github.com/tavinus/cloudsend.sh/wiki/Input-Globbing
- https://ec.haxx.se/cmdline/cmdline-globbing
CLICK HERE to see how to send an entire folder (and subfolders?) with the new script and find
Migrated to repository
https://github.com/tavinus/cloudsend.sh
This gist will not be updated anymore.
Download with curl
To
current folder
To
/usr/local/bin/cloudsend
Using sudo for sys install