linux.md

Simple things

Output to console string which goes to bc, and result from bc
echo "2+3" | tee /dev/tty | bc

Remove country specific characters and [^a-zA-Z0-9._] in filename for FILE in *.doc ; do mv "$FILE" "$( echo $FILE | iconv -f UTF-8 -t US-ASCII//TRANSLIT - | sed -r 's/[^a-z0-9._]+/_/ig' )" ; done

Split file to 'n' lines:
split -l <lines> <input> <prefix>

SSH

Reverse ssh connection, execute from remote, connect to local 2200
ssh -R 2200:localhost:22 user@remote

How to Ms Excel to linux csv utf8

1. Save worksheet as a unicode text (*.txt)
2. iconv -f utf16 -t utf8 -o out.txt input.txt
3. cat office_users_2017_05_utf8.txt | tr '\t' ';' > office_users_2017_05_utf8.csv

Redirect all outputs

foo > stdout.txt 2> stderr.txt
foo > allout.txt 2>&1

Redirect all outputs with tee

https://stackoverflow.com/questions/692000/how-do-i-write-stderr-to-a-file-while-using-tee-with-a-pipe
Old way:
./aaa.sh 2>&1 | tee -a log

Bash 4.0:
./aaa.sh |& tee -a log

To make separate file for stdout and stderr:
command > >(tee -a stdout.log) 2> >(tee -a stderr.log >&2)

Let's split it up and explain:
> >(..)
>(...) (process substitution) creates a FIFO and lets tee listen on it. Then, it uses > (file redirection) to redirect the STDOUT of command to the FIFO that your first tee is listening on.

Same thing for the second:
2> >(tee -a stderr.log >&2)
We use process substitution again to make a tee process that reads from STDIN and dumps it into stderr.log. tee outputs its input back on STDOUT, but since its input is our STDERR, we want to redirect tee's STDOUT to our STDERR again. Then we use file redirection to redirect command's STDERR to the FIFO's input (tee's STDIN).

Process substitution is one of those really lovely things you get as a bonus of choosing bash as your shell as opposed to sh (POSIX or Bourne).

In sh, you'd have to do things manually:

out="${TMPDIR:-/tmp}/out.$$" err="${TMPDIR:-/tmp}/err.$$"
mkfifo "$out" "$err"
trap 'rm "$out" "$err"' EXIT
tee -a stdout.log < "$out" &
tee -a stderr.log < "$err" >&2 &
command >"$out" 2>"$err"

Get milisec

date +%s returns the number of seconds since the epoch.
date +%s%N returns the number of seconds + current nanoseconds.
$(($(date +%s%N)/1000000)) returns the number of miliseconds

SSH File System

Mount

sshfs user@server:/remote/path /local/path -C -p 1234
-C - compression
-p 1234 - port

Unmount

fusermount -u /local/path

Map users

sshfs myuser@mycomputer:/remote/path /local/path -o idmap=user
-o idmap=user - map local user to remote myuser (only UID, not GID)

Change java version

archlinux-java <COMMAND>

COMMAND:
	status		List installed Java environments and enabled one
	get		Return the short name of the Java environment set as default
	set <JAVA_ENV>	Force <JAVA_ENV> as default
	unset		Unset current default Java environment
	fix		Fix an invalid/broken default Java environment configuration

Record audio / video

Video

ffmpeg -video_size 1920x1080 -framerate 30 -f x11grab -i :0.0+1280,0 -c:v libx264 -qp 0 -preset superfast -crf 17 capture.mp4
-i - screen and shift
-qp 0 - lossless quality
-present superfast - faster more fps and bigger file, slower less fps and slower file
-crf 17 - lower is higher quality, 17 is almost lossless; 0-51

Audio

arecord -f cd -t raw | lame -x -r – out.mp3

Video + audio

ffmpeg -f x11grab -video_size 1920x1080 -framerate 30 -i :0.0+1280,0 -f alsa -ac 1 -ar 44100 -i hw:2,1 -c:v libx264 -qp 0 -preset superfast -crf 17 capture.mp4
-i- - hardware from arecord -l (card and device)

Amplify audio

ffmpeg -i inputfile -vcodec copy -af "volume=10dB" outputfile

find

delete specific type of file (here symlinks)

Modern find version:
find -type l -delete
Old way: find -type l -exec rm {} \;

sed

change specific letter to uppercase with backreference and \u

sed -e 's/\([a-z]\)/\u\1/' input

sqlite

import csv file

$ sqlite3 users.sql
SQLite version 3.21.0 2017-10-24 18:55:49
Enter ".help" for usage hints.
sqlite> .mode csv
sqlite> .import users.csv users
sqlite> .schema users
CREATE TABLE users(
  "id" TEXT,
  "ItemId" TEXT,
  "name" TEXT
);
sqlite> select * from users limit 5;

Resize partition and filesystem

1. fdisk /dev/sdX a. check starting point of partition, if <2048 then run fdisk with -c=dos b. delete partionion and create new. Be careful about starting point c. write changes to disk
2. reboot or do partprobe /dev/sdX
3. expand filesystem resize2fs /dev/sdX1

log size and set max size

journalctl --disk-usage - check log size
journalctl --vacuum-size=512M - reduce size of logs
journalctl --verify - verify that after vacuum logs are not corrupted
To set max size of logs, edit /etc/systemd/journald.conf and add/change this value:
SystemMaxUse=512M

pacman

Removing unused packages (orphans)

For recursively removing orphans and their configuration files: pacman -Rns $(pacman -Qtdq)
If no orphans were found, pacman errors with error: no targets specified. This is expected as no arguments were passed to pacman -Rns.

Listing packages by size

pacgraph -c | sort -h

sed

delete

Remove 3th line:
sed '3d' fileName.txt

Remove the interval between lines 7 and 9:
sed '7,9d' filename.txt

Remove the line containing the string "awk," by using:
sed '/awk/d' filename.txt

You can remove the last line by typing in:
sed '$d' filename.txt

Remove all empty lines through:
sed '/^$/d' filename.txt