fanurs

1. Git clone the repo

git clone https://github.com/fanurs/data-analysis-e15190-e14030.git

2. Make sure that conda is available. This means that you should have activated your anaconda or miniconda installations. To check,

which conda

If you see some path to conda, then it's good. If not, that means somehow it's not activated yet. You could do something like:

fanurs

First, create a local directory that you want to mount the remote filesystem:

mkdir remote-mount

Assume that SSH configuration has already been set up for the remote server.

sshfs -o ro,default_permissions remote:/some/target/directory remote-mount

fanurs

Setup rclone for Google Drive on a Remote Linux Server

Installation

Install rclone:

sudo apt update
sudo apt install rclone

Configuration

fanurs

#!/bin/bash
TIMEOUT=60 # in seconds
TIMESTEP=3 # in seconds
JOB_NAME="tunnel" # any unique name

SCRIPT="/path/to/interactive.sbatch" # modify accordingly
# It starts an OpenSSH server to enable interactive sessions, allowing IDEs like VSCode to connect remotely.
# Here is an example script: https://crc-pages.pitt.edu/user-manual/slurm/vscode/#steps-performed-only-once

main () {

fanurs

env_prompt: '($(basename {default_env})) '
channels:
   - defaults
   - conda-forge
pkgs_dirs:
   - /optional/directory/in/case/your/home/directory/is/small/.conda/pkgs

fanurs

gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.5 -dPDFSETTINGS=/default -dNOPAUSE -dQUIET -dBATCH -dDetectDuplicateImages -dCompressFonts=true -dDownsampleColorImages=true -dColorImageResolution=200 -dDownsampleGrayImages=true -dGrayImageResolution=250 -sOutputFile=compressed_main.pdf

fanurs

# SSH after version 7.5 provides ProxyJump (-J flag)
# name: your username in the remote server
# when using Windows PowerShell, you may need to replace "ssh" by "ssh.exe"
ssh -J name@intermediateserver name@remoteserver

# In the case of multiple intermediate hosts, simple join all of them with comma
ssh -J name@intermediate1,name@intermediate2,name@intermediate3 name@remoteserver

# SSH before version 7.5 would need to use ProxyCommand
ssh -o ProxyCommand="ssh -W %h:%p name@intermediateserver" name@remoteserver

fanurs

#!/bin/bash

#############################
# from "here" to "there"
#############################
scp source_files user@destination:/path/at/the/destination

# with jumphost
scp -o 'ProxyCommand ssh user@jumphost -W %h:%p' source_files user@destination:/path/at/the/destination

fanurs

#!/usr/bin/env python3
import argparse
import functools
import http.server
import threading

class ArgumentParser:
    def __init__(self):
        self.parser = argparse.ArgumentParser()
        self.parser.add_argument("port", type=int, default=9090, help="port to listen on")

fanurs

import matplotlib as mpl
import matplotlib.pyplot as plt

# some pseudo-data (incomplete)
df1 = ...
df2 = ...

fig, ax = plt.subplots()
ax.errorbar(df1.x, df1.y, yerr=df1.yerr, label='df1')
ax.errorbar(df2.x, df2.y, yerr=df2.yerr, label='df2')