Create a gist now

Instantly share code, notes, and snippets.

@fspaolo /remote.md
Last active Dec 19, 2015

Working on remote machines

Some (trivial?) commands to work on a remote machine using the SSH, FTP and HTTP protocols.

SSH certificate

Generate SSH keys for using a SSH connection without having to enter the password every time you log in.

  1. Generate the key (local machine):

    $ cd ~/.ssh $ ssh-keygen -t rsa -C "username@youremail.com" $ chmod 700 id_rsa

if .ssh does not exist create it: mkdir ~/.ssh. ssh-keygen will create two files with the private and public keys: id_rsa and id_rsa.pub. Skip this step if you have already the keys in your machine!

  1. Log on the remote host (remote machine):

    $ ssh username@remote.host.edu $ cd ~/.ssh

if .ssh does not exist create it: mkdir ~/.ssh.

  1. Copy your public key (local machine):

    $ cat ~/.ssh/id_rsa.pub | pbcopy

pbcopy will copy the content to the Clipboard.

  1. Add the public key to the remote host (remote machine):

    $ pbpaste > ~/.ssh/authorized_keys # to create the file $ chmod 644 authorized_keys

pbpaste will paste the content from the Clipboard.

Log on the remote machine without entering the password. If necessary, give the path to your private key (commonly not needed!):

$ ssh -i ~/.ssh/id_rsa username@remote.host.edu

Read more about SSH keys.

Restart/Shutdown the computer

Restart a local computer immediately:

$ sudo shutdown -r now

Restart a remote computer at a specific time:

$ ssh -l root computer shutdown -r hhmm

Shut down a remote computer in 30 minutes:

$ ssh -l root computer shutdown -h +30

Job control

A job (or process) is defined as an "instance" of an executing program.

List the jobs in your job table:

$ jobs

Kill job number [2] on the list:

$ kill %2 

Display information about your processes running:

$ ps

Some ps options::

-f  Displays full information.
-e  Displays all processes running.
-u *username*  Displays user processes including those from other sessions.

Kill process by PID number:

$ kill 3682

Note: if you try to logout and get the message

There are stopped jobs

list the jobs (using jobs) and kill the stopped jobs or bring them to the foreground (see bellow), or simply type: logout logout (yes, twice).

Run a job in the background with &:

$ nohup commands &

nohup enables the command to keep running after the user has logged out. The output that would normally go to the terminal goes to a file called nohup.out if it has not already been redirected (e.g., > file.out).

Bring a background (or stopped) job to the foreground:

$ fg %jobnumber

Place a foreground job in the background (to free the terminal):

$ ^Z    # type Control-z to suspend de job
$ bg

Monitor (in "real time") the main processes running and information about system/hardware usage (e.g., CPU, memory, network, etc.):

$ top

Read more about job control.

File transfer via SSH

scp is an application to copy files to, from, or between different hosts. It uses SSH for data transfer and provides the same authentication and same level of security as SSH.

rsync is an application to analyse files and only copy the changes made to files rather than all files. See option bellow for SSH transfer protocol. The rest, similar syntax as scp.

Copy file from remote host to local host:

$ scp username@remote.host.edu:/remote/file.ext /local/directory

Copy file from local host to remote host:

$ scp file.ext username@remote.host.edu:/remote/directory

Copy file from remote host1 to remote host2:

$ scp username@remote.host1.edu:/remote/file.ext username@remote.host2.edu:/remote/directory

Copy multiple files from local host to remote host:

$ scp file1.ext file2.ext files3.ext username@remote.host.edu:/remote/directory

Note: for more than a few files, as scp spawns a new process for each file, use rsync instead.

Some scp options::

-r  Recursively go through directories.
-C  Compress the data before it goes over the network.

Note: -r does not know about symbolic links and will blindly follow them.

Some rsync options::

-a  Archive mode, preserves file permissions and does not follow symlinks.
-z  Enable compression. Compress each file as it gets sent through the pipe.
-e ssh  Uses SSH as the transport.
-v  Verbose, lists files being copied.

Obs: use of trailing slashes can be confusing.

File transfer via FTP

Interactive

ftp is an application (client) to transfer files between computers connected via the File Transfer Protocol (FTP).

Connect to the remote host:

$ ftp remote.host.edu

For a public FTP use

user: username@youremail.com
password: anonymous

Change dir and see content:

$ cd dirname
$ ls

Copy file from remote host to local host (initial local dir):

$ get remotefile.ext

Copy file from local host (initial local dir) to remote host:

$ put localfile.ext

Copy multiple files (to/from initial local dir):

$ mget *.ext
$ mput *.ext

For not answering (Y/N) type: prompt off.

Close the connection:

$ quit

Non-interactive

For non-interactive download you can use wget or curl.

List directory:

$ wget ftp://ftp.host.name.edu/pub/directory/    # this will generate an index.html file
$ open index.html                                # whole directory structure listing

or

$ curl ftp://ftp.host.name.edu/pub/directory/

Put the trailing slashes on directories.

Get files:

$ wget ftp://ftp.host.name.edu/pub/directory/*.ext

or

$ curl ftp://ftp.host.name.edu/pub/directory/file.ext -O    # no globbing

Some wget options:

--ftp-user=user  Specifies the username.
--ftp-password=password  Specifies the password.
-r  Recursive download.
-m  Keep a mirror of a directory (-r -N -l).
-c  Resume getting a partially-downloaded file.
-A  Comma-separated list of accepted extensions.
-R  Comma-separated list of rejected extensions.

Some curl options:

--user username:password  Specify user and password.

This is non-recursive!

File transfer via HTTP

Download a whole website keeping the original structure (mirror):

$ wget -m http://www.website.com/

Download recursively all .html files from a website directory:

$ wget -r -A.html http://www.website.com/directory/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment