Skip to content

Instantly share code, notes, and snippets.

@ruario
Last active January 19, 2022 17:22
Show Gist options
  • Save ruario/1247bba88b4d1753073f to your computer and use it in GitHub Desktop.
Save ruario/1247bba88b4d1753073f to your computer and use it in GitHub Desktop.
How to upload a file(s), to a remote location in one go with sftp. Useful for automation.

Normally if you want to quickly send a file to a remote directory on an ssh server as a one liner or within a script, the following would suffice:

scp file server:path

Occasionally, you may come across a server that only has sftp enabled and not scp. For example if the OpenSSH server you were connecting to was configured as follows:

Subsystem sftp internal-sftp

Match User yourusername
  ChrootDirectory /home/%u
  ForceCommand internal-sftp

Then attempting to connect via scp, would result in the error message: "This service allows sftp connections only."

In such a case, you can simulate basic scp behaviour as follows:

echo put file | sftp -b- server:path

Alternatively, if you need to display the progress of the file as it uploads:

printf "progress\nput file" | sftp -b- server:path

If you need to upload multiple files, you could use a for loop:

for f in file1 file2 file3; do echo put "$f" | sftp -b- server:path; done

(If this example file* would also have worked as a wildcard in place of file1 file2 file3).

You can combine all of the above to make a very simple upload script like so:

#!/bin/sh
s="$1"
shift 1
for f in "$@"; do
  printf "progress\nput $f" | sftp -b- "$s" || exit 1
done

Give this a suitable name (e.g. upload-sftp) and use it as follows:

upload-sftp server:path file*

Using Bash instead of Bourne Shell would allow you to specify server:path as the last option (like scp), i.e.:

upload-sftp file* server:path 

To do that, use the following:

#!/bin/bash
eval s=\$$#
for f in "${@:1:$(($#-1))}"; do
  printf "progress\nput $f" | sftp -b- "$s" || exit 1
done

Note: Using sftp in this way requires non-interactive authentication using keys to be configured. You can set -oBatchMode=no to use password authentication but this would be cumbersome when uploading multiple files using the above example, as you would be prompted for a password before each file is uploaded.

P.S. Yes I know the man page specifically mentions the -b switch but some basic examples help.

✍ 2022-01-17

Years later I needed something like this again. See my 'phlog' (gopher://sdf.org/1/users/r0/phlog) posts for details.

  1. gopher://sdf.org/0/users/r0/phlog/2022-01-14_One_shot_sftp.txt
  2. gopher://sdf.org/0/users/r0/phlog/2022-01-16_One_shot_part2.txt

Don't think you have a gopher client? You might… you can use lynx or even curl! Indeed, gopher is such a simple protocol, that if you have netcat (nc) or telnet installed, you could even hack them into fetching the content…

echo /users/r0/phlog/2022-01-14_One_shot_sftp.txt | nc sdf.org 70
(echo /users/r0/phlog/2022-01-16_One_shot_part2.txt; cat -) | telnet sdf.org 70

Alternatively, if you can't be bothered with all that or find the thought process of how I arrived at this boring, here is a summary.

#!/usr/bin/env -S bash -eu
if [ "${1:-}" = '-P' ]; then
  p="${@:1:2}"
  shift 2
fi
eval l=\${$#}
if [[ "$l" == *:* ]]; then
  for f in "${@:1:$(($#-1))}"; do
    printf '%s\n' "put ${f// /\\ }"
  done | sftp ${p:-} "$l"
else
  f="${1#*:}"
  printf '%s\n' "get ${f// /\\ } \"$2\"" | sftp ${p:-} "${1%%:*}"
fi

Limitations

  • You can't download multiple files like so, server:\{file1,file2\} .—I am not sure how many people knew about that scp feature anyway. You can still grab more than one file at a time using wild cards however, e.g. server:file\?.txt .
  • You can't download to a directory with colon in the name, or 'on-the-fly' rename a download to filename that contains a colon.
  • You can't rename a file on upload like so, file1 server:file2—you can still copy to an already present remote directory, e.g. file1 server:dir.
  • Only the -P (port) option is supported and (if you want to use it) it must be the first argument with a space to separate it from the port number.
  • You can't do server to server copies—another feature that might not even be widely known or used.
  • If certain unusual characters actually appear in filenames—and are not intended as wildcards—they would need to be double escaped (e.g. a file named file*1, would need to be uploaded as file1 server:"file\*1").
@Lex-2008
Copy link

Lex-2008 commented Dec 4, 2014

i would actually think (but i've never tried) that you can upload multiple files in one sftp session, like this:

for f in file1 file2 file3; do echo put "$f"; done | sftp -b- server:path

That would help if you have to use password authentication and want to avoid entering your password for every file.

@yougg
Copy link

yougg commented Dec 12, 2016

upload multiple files by one line:

sftp {user}@{server}:{remote_path} <<< "put {localfiles}"

@ruario
Copy link
Author

ruario commented Jan 17, 2022

Hello, 6 years later 😜

@Lex-2008 It might 'seem like' that should work without re-authentication but it does not. 😉

@yougg That is essentially the same as my first example but less portable as here-strings do not work in POSIX shells. Still… nice to have an alternative. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment