Skip to content

Instantly share code, notes, and snippets.

@peterwittek

peterwittek/cheat_sheet.md

Last active July 15, 2021 13:15
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save peterwittek/fcd7cf394278c9aad4e3f20202daf3b4 to your computer and use it in GitHub Desktop.
Save peterwittek/fcd7cf394278c9aad4e3f20202daf3b4 to your computer and use it in GitHub Desktop.
Download ZIP
Cheat sheet for QIT01
Raw
cheat_sheet.md

Hardware

  • CPU: Intel Xeon E5-1650 v4, 6 physical cores @ 3.60-4.00GHz
  • RAM: 128GByte
  • GPU: Tesla K40, 2880 physical cores @ 745-875MHz,
  • GRAM: 12GByte

Logging in

If you connect to the eduroam wifi within ICFO, you should be able to log in to any computer within ICFO. Otherwise, you have to use the VPN (e.g., from the open wifi network or from home).

ssh icfo_username@qit01.icfo.es

On Windows, you can use putty as an ssh client.

Do this once

Once you are logged in:

echo ". /export/bash_custom" >> ~/.bashrc

Then log out and log in again. This sets up the path and other environment variables correctly.

Using tmux

If your ssh connection breaks, you lose your calculations. To avoid this, you want to be able to detach from a session. The utility tmux lets you do this. Run it by

tmux

Then do something, for instance

ls -al

Now detach the session with CTRL+b d. You can log out if you want to. Then, whenever you feel like, you can reattach to your tmux session by

tmux attach

This is the most elementary use of tmux. You can also use it to multiplex several sessions: it is like having tabs in a browser. In tmux, use CTRL+b c (c for create) to open a new "tab". Go back and forth between your tabs by CTRL+b p and CTRL+b n (stands for previous and next).

Be nice

Literally, be nice:

nice -n 19 whatever_you_want_to_run

Nice levels range from -20 (top priority) to 19 (lowest). Processes start with nice 0 by default. If you run a long parallel computation with nice 0, everybody else will suffer: we do not have a queue system on this cluster.

If you love queues, go for cheetah.icfo.es. This is the log-in node for the QOT cluster. Instructions are here.

If you manage to crash the system by using more than 128Gbytes of memory, we have access to a supercomputer with four times more main memory.

Do a clean-up occasionally. Only you, ICFO IT, and a reboot can kill your processes. Check what you are running:

ps -u icfo_username

Kill'em all:

killall -s 9 name_of_process

Or just kill a single process:

kill -s 9 pid

Where pid is the process ID of the process. Another way to kill a process is to press k in top and choose which process to kill and with what signal. The signal 9 is the strongest: this immediately terminates the process at the operating system level. A softer kill is signal 15, but if you have to resort to manual killing, you probably want to use signal 9 anyway.

What's there

SDP solvers:

  • SDPA (both regular sdpa and the arbitrary-precision variant sdpa_gmp).
  • Mosek (licence is granted, version 8).
  • Solvers you do not want to use: cvxopt, cvxpy.
  • SCS (first-order method, GPU enabled).

Languages:

  • Python 3 (Anaconda variant, default) and 2 (if you hate life).
  • Julia 0.5.
  • GCC 4.8 (default) and 5.4.
  • CUDA 8.0 (this is what you need to compile things for the GPU).
  • Haskell.

Other things you might need:

  • git.
  • Vertex enumeration and other exciting tasks with polytopes: cdd and plrs. They also work at arbitrary precision.
  • TensorFlow and another handful of deep learning frameworks (GPU enabled).
  • MPI.

What's not there

Proprietary software other than the CUDA drivers and Mosek. There is strictly no MATLAB or Mathematica. We do not have an ICC licence either.

Examples

Simple one

Suppose someone puts a computational appendix online to supplement a paper, or writes some educational material like this one. Then you can convert it to a script and run it rightaway:

wget https://raw.githubusercontent.com/peterwittek/ipython-notebooks/master/Quantum_Bound_on_CHSH.ipynb
jupyter nbconvert --to script Quantum_Bound_on_CHSH.ipynb Quantum_Bound_on_CHSH.py
python Quantum_Bound_on_CHSH.py

Another one that takes some time

Everything under /export is user-writeable. You can put calculations there that you want to share with the rest of the group (the files in your home folder can only be accessed by ICFO IT). Open a tmux window and run this:

nice -n 19 python /export/example/hubbard.py

Open another tmux window and study top:

top

Convenience

Passwordless log in

Following these instructions, generate a pair of authentication keys locally:

ssh-keygen -t rsa

Then make a .ssh folder on the server:

ssh icfo_username@qit01.icfo.es mkdir -p .ssh

Finally, copy the public key to the server:

cat .ssh/id_rsa.pub | ssh icfo_username@qit01.icfo.es 'cat >> .ssh/authorized_keys'

Rejoice, you will never be prompted for a password again:

ssh icfo_username@qit01.icfo.es

Mounting remote file system

Ubuntu: File Manager->File...->Connect to server

Normal Linux:

sshfs username@qit01.icfo.es: MOUNT_POINT -C

When you are done, unmount it with

fusermount -u MOUNT_POINT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment