jimyang2008/tips.md

## tips.md

      
    Raw
  

              tips.md
            
          
    workaround for unknown trust error in pacman -S

change SigLevel in /etc/pacman.conf
#SigLevel    = Required DatabaseOptional
SigLevel = Optional TrustAll

configure motion

$ cat /etc/motion/motion.conf
video_device /dev/video0
width 1280
height 720
target_dir /var/run/motion
pause on
movie_output off
snapshot_interval 600
webcontrol_port 8111
webcontrol_localhost off
stream_port 8112
stream_localhost off
stream_quality 100

fix keyring in ArchLinux

killall gpg-agent
rm -rf /etc/pacman.d/gnupg/
pacman-key --init
pacman-key --populate archlinux

upgrade Arch Linux to latest

# update system clock to correct time
$ sudo date -s '2024-04-01 08:06:46'

# update keyring
$ sudo pacman -Sy archlinux-keyring

# upgrade archlinux
$ sudo pacman -Syyu

set advertised speed in ethernet port

# advertise ALL speeds, note this might NOT work on all ethernet ports
sudo ethtool -s eth0 advertise 0x1800000001028

load kernel module nt_nat_tftp to allow TFTP client connect to remote TFTP server

sudo modprobe nf_nat_tftp
sudo iptables -t raw -I PREROUTING -j CT -p udp -m udp --dport 69 --helper tftp

Use tshark - command line replacement of wireshark

# set wlan0 to monitor mode on 5G channel 40(5.2GHz) before capture 
sudo ip link set wlan0 down
sudo iw wlan0 set monitor none
# or, sudo iwconfig wlan0 mode monitor
sudo ip link set wlan0 up
sudo iwconfig wlan0 channel 40
# or, sudo iw wlan0 set freq 5200 80 5210
sudo iw wlan0 info

# capture with tshark
sudo tshark -n -i wlan0 -V -Y 'wlan.ext_tag.number == 107' 'type mgt subtype beacon'
 - `-i` wlan0 : capture interface wlan0
 - `-n` : 
 - `-V` : show packet detail
 - `-Y` ... : display filter
 - `type mgt subtype beacon`: capture filter

To avoid ARP flux

  sysctl -w net.ipv4.conf.all.arp_announce=1
  sysctl -w net.ipv4.conf.all.arp_ignore=2

To allow systemd-resolved listen to LAN for DNS requests

Add extra listening address:port in /etc/systemd/resolved.conf
DNSStubListenerExtra=192.168.213.1:53

Schedule execution in Linux


cron


at


echo xxx | at 12:34


systemd-run( .timer/.service )

--on-calendar
--on-active=1s --on-unit-active=100s


simple sleep (not accurate on interval time between starting points)


sleep with wait (more accurate)


sleep $interval &
sleep_pid=$!
...
wait $sleep_pid

numeric conversion between DEC and HEX in bash


DEC to HEX
printf '0x%x\n' 39


HEX to DEC
echo $((0x27))


reset tty size in screen connected console window

* check tty size before entering "screen" to get actual window size
stty size
* resize inside "screen" since it cannot detect window size
stty rows 72 cols 270

capture command output with filtering in file and leave it in console as well

# use process substitution
stdbuf -oL script | tee >(grep --line-buffered pattern > /path/to/log)

# or use named pipe
## in terminal 1
mkfifo /path/to/named_pipe
stdbuf -oL script > /path/to/named_pipe
## in terminal 2
cat /path/to/named_pipe
# or
tail -n +1 -f /path/to/named_pipe

Use SOCKS5 proxy in Git

  ssh -fnND 7070 -p $port $user@$remote_host
  git config --global http.proxy 'socks5://127.0.0.1:7070'
  git config --global --unset http.proxy

Serve static file(script) for HTTP download in one line


server
cd /dir/of/script/
python -m SimpleHTTPServer


client
curl http://<server_ip>:8000/<script_name>


Switch USB mode from DISK to NIC for some WiFi USB adapter

# use lsusb to get vid:pid
sudo apt install -y usb-modeswitch
sudo /usr/sbin/usb_modeswitch -K -v 

Enable debug in wireguard

sudo bash -c 'echo module wireguard +p > /sys/kernel/debug/dynamic_debug/control'
journalctl -kf

Bypass certificate check in git

export GIT_SSL_NO_VERIFY=1
Keep current script output to log

exec &> >(tee /path/to/logfile)
kill processes causing loop device busy and failed to umount

  sudo fuser -km /mnt/point
  sudo umount -R /mnt/point

disable naming scheme

  # update /etc/default/grub
  GRUB_CMDLINE_LINUX_DEFAULT="net.ifnames=1 biosdevname=0"

Add route in specific table


show ip rules with tables
sudo ip rule list


check route for given target IP
sudo ip route get $DST_IP from $SRC_IP iif $SRC_INTERFACE


add route in given table (default="main")
sudo ip r add 192.168.104.0/24 dev eth0 table eth0_local


Change screen resolution in grub

  # find out supported resolutions in running system
  $ sudo hwinfo --framebuffer
  # find out supported resolutions in GRUB boot menu
  > videoinfo
  # edit /etc/default/grub and update
  # For example,
  #   GRUB_GFXMODE=1920x1080x32,auto
  #   GRUB_GFXPAYLOAD_LINUX=keep
  $ sudo vi /etc/default/grub
  $ sudo update-grub

How to compile a single kernel module for a device driver with given source code

  $ cd /path/to/devcie_driver_source/
  $ make -C /lib/modules/$(uname -r)/build M=$(pwd) modules

How to add static route in case of multiple routing tables

# get first routing table used on eth1
$ ip rule list | awk '$5=="eth1" {print $NF; exit}'
eth1_local

# add static route in located table
$ sudo ip r add 192.168.53.195/32 via 192.168.53.1 table eth1_local

# show routes in given table
$ ip route show table eth1_local
192.168.53.0/24 dev eth1 scope link src 192.168.53.165
192.168.53.195 via 192.168.53.1 dev eth1

Using Git


Check remote tags
git ls-remote --tags <repo_url>


Checkout a single branch by tag
git clone --depth 1 --single-branch --branch <tag_name> <repo_url>


Squash commits for a PR


$ git fetch upstream
$ git checkout master
$ git rebase -i upstream/master
$ git push origin master -f

How to activate Windows 10 with private KMS(using vlmcsd)


Start a KMS server with VLMCSD (http://wind4.github.io/vlmcsd/)

# get vlmcsd source code
git clone https://github.com/Wind4/vlmcsd`
# build to get `vlmcsd` binary
cd vlmcsd
make
# start VLMCSD
bin/vlmcsd -L $LAN_IP


Activate in Windows 10 against above VLMCSD

# Get a product key at https://docs.microsoft.com/en-us/windows-server/get-started/kms-client-activation-keys
# Start a CMD with admin priviliege
# Register the product key in Windows
slmgr /ipk <product_key>
# Set KMS server address
slmgr /skms <vlmcsd_LAN_IP>
# Activate with
slmgr /ato

How to prepare a Windows 10 USB installer in Windows


Get a Windows ISO by either of following ways

Download from https://itellyou.cn/
Dump from a running Windows using tool https://www.microsoft.com/en-us/software-download/windows10


Plug in a 8GB+ USB drive or SD card+USB card reader
Use rufus

Select image in rufus (DO NOT drag and drop file to it)
Select GPT, UEFI(no CSM), and then start


netconsole

Debug kernel panic on remote host
# load netconsole kernel module on host
#.  sudo modprobe netconsole netconsole=<sender_port>@<purple_ip_address/eth0,<receiver_port>@<receiver_ip>/<next_hop_mac_address>
sudo modprobe netconsole netconsole=6666@192.168.165.154/eth0,6666@192.168.165.1/20:6d:31:01:04:40
sudo bash -c "echo 1 > /proc/sys/kernel/sysrq"
sudo bash -c "echo c > /proc/sysrq-trigger"

# start listening to UDP port on debug host
#.  nc -u -l -p <receiver_port>
nc -u -l -p 6666


nmap


scan for SSHable host in IP range
sudo nmap -sT -p 22 192.168.69.0/24 | grep report


openconnect


server install

sudo apt install -y ocserv
sudo ocpasswd pi
# set auth = plain[password=/etc/ocserv/ocpasswd]
sudo vi /etc/ocserv/ocserv.conf


client install

sudo apt install -y openconnect
sudo tee /usr/lib/tmpfiles.d/openconnect.conf <<EOF
d /run/vpnc 0710 root root -
EOF


client build

sudo apt update
sudo apt install -y libxml2-dev zlib1g-dev libssl-dev autotools-dev automake libtool liblz4-dev
git clone git://git.infradead.org/users/dwmw2/openconnect.git
cd openconnect/
./autogen.sh
./configure --without-gnutls --disable-nls
make
sudo make install

APT install hangs in chroot environment

Do not mount /run at all. That's the cause. But no idea why.
Upgrade Debian9(Stretch) to Debian10(Buster)

# update Debian9 to latest
sudo apt update
sudo apt -y upgrade
sudo apt -y full-upgrade
sudo apt --purge autoremove

# change APT sources.list from `stretch` to `buster`
sudo sed -i -e 's/stretch/buster/g' /etc/apt/sources.list

# update Debian10 to latest
sudo apt update
sudo apt -y upgrade
sudo apt -y full-upgrade

sudo reboot

Change default CPU affinity for worker thread by setting cpumask of workqueue

 #  only allow 4 out of 6 cores(11 1100) being used by worker threads
 echo 3c > /sys/devices/virtual/workqueue/cpumask

chroot from x86_64 host to aarch64 rootfs

# copy QEMU cross platform binary in place
cp /usr/bin/qemu-aarch64-static $ROOTFS_DIR/usr/bin/

# mount for chroot
# !!IMPORTANT: DO NOT do `mount -B /run $ROOTFS_DIR/run` as normal chroot !!
mount -t proc chproc $ROOTFS_DIR/proc
mount -t sysfs chsys $ROOTFS_DIR/sys
mount -t devtmpfs chdev $ROOTFS_DIR/dev || mount --bind /dev $ROOTFS_DIR/dev
mount -t devpts chpts $ROOTFS_DIR/dev/pts

# chroot as usual
chroot $ROOTFS_DIR /bin/bash

one-liner to check ethernet queuing and CPU affinity settings

tail /sys/class/net/eth*/queues/rx-0/rps_{cpus,flow_cnt} $(eval echo /proc/irq/{$(awk -F: '/[0-9]{4,} .* eth/ {print $1}' /proc/interrupts|paste -sd ,|tr -d ' ')}/smp_affinity) | sed '/^$/d'

traffic control with queuing discipline

# show current `tc qdisc` settings
sudo tc qdisc show dev eth0
# to add delay and remove it
sudo tc qdisc add dev eth0 root netem delay 100ms
sudo tc qdisc del dev eth0 root netem delay 100ms
# to drop packets and remove it
sudo tc qdisc add dev eth0 root netem loss 30%
sudo tc qdisc del dev eth0 root netem loss 30%
# to limit bandwidth
sudo tc qdisc add dev ens5 root tbf rate 1024kbit latency 50ms burst 1540


Change ethernet interface name in udev

Refer to

systemd-udevd
systemd.link
udevadm

# collect udev parameters on given interface with udevadm
sudo udevadm info -a /sys/class/net/eth0
# use combinations of found parameters to locate given device
$ cat /etc/udev/rules.d/70-persistent-net.rules
# Each interface must be in a single line
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*",ATTR{address}=="02:01:02:03:04:05",ATTR{dev_id}=="0x0",ATTR{type}=="1",NAME:="eth0"

For example, given
  looking at parent device '/devices/pci0000:00/0000:00:13.0/0000:01:00.0':
    KERNELS=="0000:01:00.0"
    SUBSYSTEMS=="pci"
    DRIVERS=="igb"
    ATTRS{enable}=="1"
    ATTRS{driver_override}=="(null)"
    ATTRS{irq}=="22"
    ATTRS{current_link_width}=="1"
    ATTRS{subsystem_device}=="0x0000"
    ATTRS{device}=="0x1539"
    ATTRS{current_link_speed}=="2.5 GT/s"
    ATTRS{ari_enabled}=="0"
    ATTRS{local_cpus}=="f"
    ATTRS{vendor}=="0x8086"
    ATTRS{msi_bus}=="1"
    ATTRS{max_link_speed}=="2.5 GT/s"
    ATTRS{revision}=="0x03"
    ATTRS{local_cpulist}=="0-3"
    ATTRS{broken_parity_status}=="0"
    ATTRS{max_link_width}=="1"
    ATTRS{numa_node}=="-1"
    ATTRS{consistent_dma_mask_bits}=="64"
    ATTRS{dma_mask_bits}=="64"
    ATTRS{class}=="0x020000"
    ATTRS{subsystem_vendor}=="0x8086"
    ATTRS{d3cold_allowed}=="1"

The udev rule can be fined as
ACTION=="add", KERNELS=="0000:01:00.0", SUBSYSTEMS=="pci", DRIVERS=="igb", ATTRS{device}=="0x1539", NAME:="eth0"

To apply, just do
udevadm control --reload

You might need to reload ethernet driver(kernel module) before reload udev
modprobe -r <ethernet_ko>
modprobe <ethernet_ko>

You can check by
udevadm test-builtin net_id /sys/class/net/<interface>

Since ethernet driver need to be loaded before making any adjustment, need do manual modprobe in udev boot script(/usr/share/initramfs-tools/scripts/init-top/udev) before scanning devices inside initramfs
Share existing X desktop in Ubuntu

Tiger VNC provides a so-called scarping server, which can share existing X session rather than create a new virtual one for sharing.
# create VNC password
$ vncpasswd

# install Tiger VNC scraping server
$ sudo apt install -y tigervnc-scraping-server

# start VNC scraping server using existing VNC password
$ x0vncserver -PasswordFile=~/.vnc/passwd

Easy install of openvpn


Download the helper scrpit and run it
curl -Lk https://git.io/vpn -o openvpn-ubuntu-install.sh
sudo ./openvpn-ubuntu-install.sh
# provide your one of your interface IP desired to bind


Linux file system

Things to do after cloning a VM

In addition to

change hostname with sudo hostnamectl set-hostname <new_hostname>
remove and re-create network NIC for VM
remove SSH host keys under /etc/ssh/ssh_host_key and run dpkg-reconfigure openssh-server

There are 2 ways to get a unique IP

add dhcp-identifier: mac to ethernet interface definition in YAML file under /etc/netplan/
remove /etc/machine-id and run sudo systemd-machine-id-setup to get a new one

Linux Traffic Control(tc) with Queueing Discipline


interface / qdisc / class / handle / filter

each interface has both egress(root) and ingress(ingress) for qdisc to attach to
a qdisc is simply a scheduler

classless qdisc can contain no classes, nor is it possible to attach filter to a classless qdisc

pfifo_fast
sbt


classful qdiscs can contain classes, and provide a handle to which to attach filters.

cbq
fq_codel
hbt


Linux filters allow the user to classify packets into an output queue


reference

https://tldp.org/HOWTO/Traffic-Control-HOWTO/index.html
https://man7.org/linux/man-pages/man8/tc.8.html


Linux Policy Based Routing(PBR)

Standard routing is based on traffic destination.
Policy based routing can use different routing table for different criteria of traffic.

Source
Type of traffic(QoS)
Packet size
Application protocol
More

Use ip rule command to define simple PBR rules based on supported criteria

source addresses
TOS
in/out-bound interfaces

Combined bwith netfilter MARK to define complex PBR rules
MARK_DNS=1
RT_DNS='rt_dns'
GW_DNS=10.0.0.2
# mark DNS traffic 
iptables -A PREROUTING -m mark --mark 0 -p udp --dport 53 -j mark --set-mark $MARK_DNS
# define a new routing table
echo "200 $RT_DNS" >> /etc/iproute2/rt_tables
# add routes to new routing table
ip route add default via $GW_DNS table $RT_DNS
# add policy rules
ip rule add pri 20000 fwmark $MARK_DNS table $RT_DNS

Make a script only run with one instance like a singleton

# create a file with a file descriptor automatically chosen and assigned to $lock_fd
# NOTE: NO SPACE before ">"
exec {lock_fd}> /var/lock/script.lock
# create an exclusive(-x) lock on $lock_fd and abort(-n) rather than wait if failed to lock
flock -x -n $lock_fd || exit 1

Linux kernel upgrade/downgrade in Ubuntu


To upgrade to a given version
KERNEL_VER=5.4.0-54-generic
sudo apt install -y linux-image-$KERNEL_VER


To downgrade to a given version


    KERNEL_VER_OLD=5.4.0-54-generic
    KERNEL_VER_NEW=5.4.0-56-generic
    # Make sure it is already installed in system
    sudo apt install -y linux-image-$KERNEL_VER_OL
    # Reboot with SHIFT held to get into GRUB menu to select kernel    
    sudo apt remove linux-image-$KERNEL_VER_NEW
    # Remove later kernel versions to make it latest

Setup Wifi in Ubuntu Linux

As Client

# install required software
sudo apt install -y wpa_supplicant dhclient
## scan wifi SSID channel frequency and signal quality
## NOTE: `sudo iwlist wlan0 scan` would ONLY scan for the ones with same frequency as the one that wlan0 connected to unless a `sudo iwlist scan` is done
sudo iwlist  scan 2>/dev/null  | egrep 'ESSID|Freq|Quality'  | paste -d'\t' - - - | grep SGold203 | sort -k2
# configure ESSID with password
wpa_passphrase $ssid $passwd | sudo tee -a /etc/wpa_supplicant/wpa_supplicant.conf
# connect to Wifi network
sudo /sbin/wpa_supplicant -u -s -c /etc/wpa_supplicant/wpa_supplicant.conf -i wlan0
# obtain IP after Wifi connected
sudo dhclient -i wlan0 -v

As Server

# install required software
sudo apt install -y hostapd dnsmasq

# check available channels
sudo iw list | grep -A20 Frequencies:
# configure hostapd
$ cat >/etc/hostapd/hostapd.conf <<EOF
interface=wlan0
driver=nl80211
hw_mode=a
channel=36
ieee80211d=1
country_code=CN
ieee80211n=1
ieee80211ac=1
wmm_enabled=1
ssid=mywifi
macaddr_acl=0
auth_algs=1
ignore_broadcast_ssid=0
wpa=2
wpa_passphrase=Hardtoguess!
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP
rsn_pairwise=CCMP
EOF

# configure wlan static IP
sudo ip address add 192.168.31.1/24 dev wlan0

# configure dnsmasq
$ cat >/etc/dnsmasq.conf <<EOF
server=9.9.9.9
except-interface=eth0
dhcp-range=192.168.31.50,192.168.31.150,12h
log-queries
log-dhcp
EOF

# unmask and enable hostapd
sudo systemctl unmask hostapd
sudo systemctl enable hostapd
sudo systemctl start hostapd
sudo systemctl enable dnsmasq
sudo systemctl start dnsmasq

# setup forwarding
sudo iptables -t nat -A POSTROUTING -j MASQUERADE -o eth0

# uncomment line: net.ipv4.ip_forward = 1
sudo vi /etc/sysctl.conf
sudo sysctl -p

NOTE: To try out different hostapd.conf, wlan0 need to be brought DOWN and let hostapd bring it up
sudo ip link set dev wlan0 down
sudo systemctl restart hostapd

Variable substitution in a text file

eval "cat <<EOF
$(< file.txt)
EOF
"

Shrink a file system with partition in an image file

Here's an example to resize a file system with partition to 3GB(a precheck shows roughly 2.5GB of disk usage).
# enlarge an image file
sudo dd of=x.img bs=26214400 seek=607 < /dev/null
sudo losetup img_file -fP --show
sudo zerofree -v /dev/loop1p1
sudo e2fsck -f /dev/loop1p1
# resize file system to minimum size
sudo resize2fs -M /dev/loop1p1
# resize partition to an estimated small size enough to hold file system
sudo parted /dev/loop1
# parted> resizepart 1 3GB
# quit
sudo e2fsck -f /dev/loop1p1
# resize file system to max size inside given partition
sudo resize2fs /dev/loop1p1
sudo mount /dev/loop1p1 /mnt/p1
df -h /mnt/p1
sudo umount /mnt/p1
sudo losetup -d /dev/loop1
# truncate image to an estimated size enough to hold partition
sudo truncate -s 3G img_file
# fix GPT partition table(ONLY needed for GPT partition table)
sudo sgdisk -e img_file
# use sfdisk to enlarge #2 partition of /dev/sdb
echo ',+14GiB' | sudo sfdisk /dev/sdb -N 2

NodeJS

Best Javascript language reference

https://javascript.info/
Use map/reduce to pick keys of objects

/* NOTES
 * map does same action to each element of an array and result in a modified array
 * reduce does accumulated action to each element of an array result in an accumlated value
 */

const x=[ { id: 1, name: 'x', sex: 'f'}, { id: 2, name:'y', sex:'m' }, {id:3, name:'z',sex:'f'}]
const s=['id','name']
nx = x.map( h => s.reduce((nh,cur)=>{nh[cur]=h[cur]; return nh;},{}) )
//[ { id: 1, name: 'x' }, { id: 2, name: 'y' }, { id: 3, name: 'z' } ]

Mount .img file

Mount with losetup


Get loop device with partition setup with losetup -P
losetup -fP --show Armbian_20.05.1_Nanopi-r2s_focal_current_5.4.43_minimal.img


Check mount partition, assume /dev/loop0 was output from above command
fidks -l /dev/loop0


Mount created loop device directly
mount /dev/loop0p1 /mnt/x


Mount directly with calculated offset


Get Start offset from fdisk -l $img output
fdisk -l Armbian_20.05.1_Nanopi-r2s_focal_current_5.4.43_minimal.img


Mount with offset
mount -o loop,offset=$((32768*512)) Armbian_20.05.1_Nanopi-r2s_focal_current_5.4.43_minimal.img /mnt/x


Mac setup

map bash key bindings in iTerm2

Goto iTerm2 -> Preferences -> Profiles -> Default -> Keys, under General Left Option key -> Esc+
Goto iTerm2 -> Preferences -> Profiles -> Default -> Keys, and add a new key mapping under "Key Mappings"
 Keyboard Shortcut: <command> + .
 Action: Send Esacpe Sequence
 Esc+ : .

Do the same for Esc+b, Esc+d, and Esc+f
Get system information


Linux

uname -a


Hardware

lshw -short
lscpu
lsblk
lsusb
dmidecode -system

Keep command output with color in file

# use script command to fake a terminal for the command to output color 
$ script -qc 'command_output_with_color' colored_output.txt

# less with -r support color rendering
$ less -r colored_output.txt 

Ansible


Playbook : a list of plays
Play : map of hosts => tasks, where hosts can be filtered directly or via group from inventory, and tasks can be specified directly or via role
Inventory : A whole set of all hosts
Group : A way to group a list of hosts defined in inventory
Task : Actual change by calls to ansible modules
Role : A re-usable unit to define tasks and related meta data to be used by Play
Var : A variable to be used in task, with value sourced from host or group

- hosts:
  remote_user:
  tasks:
  - name:
    ...
  - name:
    ...


A role : way of automatically loading certain vars_files, tasks, and handlers based on a known file structure

site.yml
roles/
    role1/
        tasks/
        handlers/
        defaults/
        vars/
        files/
        templates/
        meta/
        

burn a single partition SD card close to its used size


mount the partition and get used size and then umount
use resize2fs to shrink it to a smaller size close to used actual one, may need e2fsck -f first
use fdisk to check, delete, and re-create the partition at same START sector but smaller size close to used size
use dd count=.. bs=.. if=/dev/sda |gzip -c > img.gz to dump image of close size

docker

run a ubuntu with bind mount directory on host machine READONLY

docker run -it --rm --privileged --mount type=bind,source=$(pwd),target=/data,readonly ubuntu:latest

search for system call

strace

trace library

ltrace

connect to DB prod_sim on sh-light with username/passwd=test/test

mysql -h sh-light -u test -ptest prod_sim
mysql> source ~/trash/p1_20070814.sql

mysql


change mysql password to support old client
update mysql.user set password=old_password('test') where user='test';
flush privileges;


export table data
select * from user_role into outfile '/tmp/user_role.sql';


import table data
load data local infile '/tmp/user_role.sql' ignore into table user_role;


purge mysql binary logs
purge binary logs to 'mysql-bin.010';


mysqldump data only
mysqldump --no-create-db --no-create-info ...


MySQL replication


On master

master configuration

[mysqld]
# enable binary logging with file name prefix of mysql-bin
log-bin=mysql-bin
# establish a unique server ID of 1
server-id=1


master commands

# flush all tables and block write
mysql> FLUSH TABLES WITH READ LOCK;
# obtaining replication master binary log coordiantes
mysql> SHOW MASTER STATUS;
# create a snapshot of all databases
mysqldump --all-databases --lock-all-tables --hex-blob > dbdump.db
# release the read lock
mysql> UNLOCK TABLES;


On client

slave configuration

[mysqld]
# establish a unique server ID
server-id=2
# stop slave
mysql> STOP SLAVE;


slave commands

# import data
mysql  < dbdump.db
# set up the slave to communicate with the master for replication
mysql> CHANGE MASTER TO
         MASTER_HOST='master_host_name',
         MASTER_USER='repliation_user_name',
         MASTER_PASSWORD='replication_password',
         MASTER_LOG_FILE='recorded_log_file_name',
         MASTER_LOG_POS=recorded log_position;


start slave


mysql> START SLAVE;


vim


scroll multiple window together in Vim
:set scb  <= DO THIS in each window


set diffopt to ignore white space
:set diffopt=filler,iwhite


how to define a filetype for an extension in Vim


For example, set filetype to text for *.txt
put this in file ~/.vim/ftdetect/mine.vim
au BufRead,BufNewFile *.txt     set filetype=text


color scheme in vim


change on the fly
:colorscheme myscheme


change permanently
add in ~/.vimrc: colorscheme myscheme


use customized one
cp myscheme.vim ~/.vim/colors/


change color output of ls


export dircolors to file
dircolors -p > x


modify it
vim x


execute bash script remotely in SSH command line

  ssh $remote 'bash -s' -- < ./script.sh "arg1" "arg2"

output bash ENV var syntax

dircolors -b x

openssl related


generate private key and CSR for certificate
  openssl req -newkey rsa:4096 -nodes -keyout server.key -subj '/CN=bgl-fb12.cisco.com' -out server.csr


sign a CSR with CA key and certificate
  openssl x509 -req -days 360 -in server.csr -CA ca.pem -CAkey ca.key.pem -CAcreateserial -out server.crt -sha256


generate private key and self-signed certificate in command
  openssl req -newkey rsa:4096 -keyout server.key -subj '/CN=bgl-fb12.cisco.com' -nodes -x509 -sha256 -days 365 -out server.crt


generate certificate with SAN extension
  openssl req -newkey rsa:4096 -nodes -keyout server.key -x509 -sha256 -days 3650 -extensions v3_ca -extensions SAN -config <(cat /etc/pki/tls/openssl.cnf <(printf "[SAN]\nsubjectAltName=DNS.1:bgl-fb12")) -subj '/CN=bgl-fb12.cisco.com' -out server.crt


check certificate valid period
  openssl pkcs12 -passin pass:openssl -nokeys -in client.pfx | openssl x509 -dates
  keytool -v -list -storepass openssl -keystore server.keystore | grep from
  keytool -v -list -storepass openssl -keystore trustedcerts | grep from


test certificates
  openssl pkcs12 -in client.pfx -out client.pem
  openssl s_client -connect https.company.com:7105 -cert client.pem -CAfile ca-key.pem


download certificate
  openssl s_client -connect pwxstg.compnay.com:443 -showcerts < /dev/null > cert.pem


convert certificate
  openssl x509 -outform DER -in cert.pem -out cert.der


save it in JKS
  keytool -import -noprompt -file $cert_der -alias $cert_host_alias -keystore $cert_jks -storepass $STOREPASS -trustcacerts


encrypt and decrypt text
  openssl enc cleartext
  openssl dec cleartext


encrypt and decrypt file
  openssl enc [-a] -des3 -salt -in file.txt -out file.des3
  openssl enc -d [-a] -des3 -salt -in file.des3 -out file.txt


verify certificate chain
  openssl verify -CAfile $ca_cert -untrusted $intermidiate_cert $server_cert


check HASH of certificate
  # get server's issuer hash to match against CA's hash 
  openssl x509 -in $ca_crt  -noout -subject -issuer_hash -hash
  openssl x509 -in $server_crt  -noout -subject -issuer_hash


colorise output

ESC_SEQ="\x1b["
COL_RESET=$ESC_SEQ"39;49;00m"
COL_RED=$ESC_SEQ"31;01m"
COL_GREEN=$ESC_SEQ"32;01m"
COL_YELLOW=$ESC_SEQ"33;01m"
COL_BLUE=$ESC_SEQ"34;01m"
COL_MAGENTA=$ESC_SEQ"35;01m"
COL_CYAN=$ESC_SEQ"36;01m"
echo -e $COL_BLUE"INFO: "$COL_RESET"This is an info message"
echo -e $COL_RED"An error has occured"$COL_RESET

Linux Disk System

LVM


group multiple physical partitions in a volume group
create logical volume from a volume group
mount logical volume

fdisk

fdisk /dev/sdb

create /dev/sdb2
set partition type to '8e' (Linux LVM)

fdisk /dev/sdc

create /dev/sdc2
set partition type to '8e' (Linux LVM)

create physcial volumes

pvcreate /dev/sdb2
pvcreate /dev/sdc2

show pv

pvdisplay /dev/sdb2

create volume group

vgcreate lvm_grp /dev/sdb2

show vg

vgdisplay lvm_grp

add pv to vg

vgextend lvm_grp /dev/sdc2

remove pv from vg

vgreduce lvm_grp /dev/sdc2

create logical volume

lvcreate -L1500 -ntestlv lvm_grp
mkfs -t ext3 /dev/lvm_grp/testlv
mount /dev/lvm_grp/testlv /mnt/testlv

remove logical volume

umount /dev/lvm_grp/testlv
lvremove /dev/lvm_grp/testlv

RAID

mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/sdb2 /dev/sdc2

cat /proc/mdstat

mdadm --detail --scan --verbose > /etc/mdadm.conf

mdadm --stop /dev/md0
mdadm --remove /dev/md0

Linux XFS image


Burn .iso to a DVD


Boot CAM25 with the DVD


Prepare a partition(e.g., /dev/sda2) with size > 4G and mount it( e.g.,
/mnt/linux)
mkdir /mnt/linux
mount /dev/sda2 /mnt/linux


Restore XFS filesystem to partition with following command
xfsrestore -f /mnt/cdrom/fa-1.0.xfs /mnt/linux


Setup GRUB in MBR
grub
root (hd0,1)
setup (hd0)


Show memory usage per user on Linux

for u in `who |awk '{print $1}' | sort | uniq`;do
    ps -u $u -o pid,rss,command | awk '{sum+=$2} END {print "'"$u"' :" , sum}'
done

SSH tunnel

local host == SSH ==> remote host
local/remote forwarding: open listening addr:port on local/remote host to forward access from remote/local host to target addr:port

local  - open a port on local host(listen_address:listen_port) to remote accessible target(addr:port) via SSH

ssh -g -nNT -L [$listen_addr:]$listen_port:$target_host:$target_port $remote_host

remote - open a port on remote host(listen_address:listen_port) to local accessible target(addr:port) via SSH connection

ssh -nNT -R [$listen_address:]$listen_port:$target_addr:$target_port $remote_host
# NOTE: GatewayPorts need to be set "yes" or "clientspecified" to allow non-local remote listening address

open a local dynamic port as SOCKS5 proxy

ssh -fqT -D 7117 vm-cvce-um-00f sleep 300

Find out the top 5 CPU using processes

ps -eo pcpu,pid,user,args | sort -k 1 -n -r | head -5

Find out memory usage every 5 seconds

free -m -s 5

Show memory usage every 2 seconds for 200 times

vmstat -a 2 200

change timezone

ln -s /usr/share/zoneinfo/Asia/Shanghai /etc/localtime

trim space

echo ' t e st this = ok ' | tr -d ' '

add a new drive /dev/sdb

fdisk /dev/sdb # create a partition /dev/sdb1
mkfs -t ext3 /dev/sdb1
e2label /dev/sdb1 /mount/point
echo "LABEL=/mount/point /mount/point ext3 defaults 1 2">> /etc/fstab

NFS server


add entry in /etc/exports
/export 10.10.0.0/255.255.0.0(rw)
add entry in /etc/hosts.allow /etc/hosts.deny
hosts.deny
portmap:ALL
hosts.allow
portmap: 10.10.0.0/255.255.0.0
start nfs service
service nfs start

NFS client


add entry in /etc/fstab
10.10.49.245:/export    /export                 nfs     bg,hard,intr,nosuid,rw 0 0


mkdir mount point mount
mkdir /export
mount -a


Find out which Linux distro

cat /etc/*-release
cat /proc/version

awk

awk '<pattern> { <aciton> }' <file>
awk '/abc/ { print $2 }' file
awk 'NR=2 { print $2 }' file

sed

sed -e '<addr> <cmd>' -e '<addr> <cmd>' <target>
sed -f <file> <target>

solaris system files related to hostname/IP

/etc/auto_home
/etc/defaultrouter
/etc/hostname.<if>
/etc/hosts # set ip address here
/etc/netmasks
/etc/nodename

Use wget to POST content of test.xml

NOTE: add "=" to begin of file for key value processing
wget --no-check-certificate --certificate=/home/vdeadmin/client-cert.pem --private-key=/home/vdeadmin/client-key.pem https://sh-debian1:7160/vde/session.jsp -O /dev/stdout --post-file test.xml

Use curl to POST content of test.xml

NOTE: add "=" to begin of file for key value processing
curl --insecure --cert /home/vdeadmin/client-cert.pem --key /home/vdeadmin/client-key.pem --data-binary @test.xml https://sh-debian1:7160/vde/session.jsp

use netcat to transfer a file


receiver:
nc -l $port > file.txt


sender:
nc -w3 $remote_host $port < file.txt


port scan

nc -n -z -w1 -v <ip> <port1>-<port2>

send email using sendmail

/usr/sbin/sendmail -t -i <<EOM
message body goes here

multiple lines are ok
.

EOM

turn off selinux/enforce

set "SELINUX=disabled" in /etc/selinux/config
add user with predefined password 'fishbowl'

useradd -p `openssl passwd fishbowl` yangjian

font


add font to fontconfig
cp <new_font_file> /usr/share/fonts/local/
fc-cache /usr/share/fonts/local/
fc-cache /home//.fonts/


add font to xfs
mkdir /usr/share/fonts/local/
chkfontpath --add /usr/share/fonts/local/
cp <new_font_file> /usr/share/fonts/local/
ttmkfdir -d /usr/share/fonts/local/ -o /usr/share/fonts/local/fonts.scale
service xfs reload


use xrandr to control screen


show all connected outputs
xrandr -q | grep connected | awk '{print $1}'


set an output's relative position to another
xrandr --output VGA --right-of LVDS


add new screen resolution
xrandr --output VGA --mode 1280x1024 --rate 75


change owner of host
grep user /etc/system-profile


diff & patch

cd parent_dir_of_original_dir
diff -r -c original_dir updated_dir > fix.patch
patch -d originial_dir -Np1 < fix.patch

invalidate nscd cache after host address change

this is useful for ssh
nscd -i hosts
/etc/init.d/nscd restart

send email with attachment

uuencode <file_path> <file_name> | mail -s 'subject' receiver

yum repository

yum search xyz
yum install ...

set resolution manually with gtf/xrandr

x=1280
y=1024
r=75
lcd=`xrandr | awk '/LVDS/ {print $1}'`
ext_monitor=`xrandr | awk '/VGA/ {print $1}'`
mode_line=`gtf $x $y $r | sed -n -e 's/.*Modeline //p' | tr -d '"'`
mode_name=`echo $mode_line | awk '{print $1}'`
xrandr --newmode $mode_line
xrandr --addmode $ext_monitor $mode_name
xrandr --output $ext_monitor --mode $mode_name --primary --output $lcd --left-of $ext_monitor

set opertaions of text files


union
sort -m <(sort file1) <(sort file2) | uniq


intersection of file1 and file2
sort -m <(sort file1) <(sort file2) | uniq -d
fgrep -x -f file2 file1
comm -12 <(sort file1) <(sort file2)


subtraction of file2 from file1
fgrep -vx -f file2 file1
comm -23 <(sort file1) <(sort file2)
sort -m file1 file2 file2 | uniq -u


unix file permission


display

position: 0123456789
example:  drwxrwxrwx
0: [-dl]
1~9:  [-r][-w][-xsS][-r][-w][-xsS][-r][-w][-xtT]

set

position:      0123
example: chmod 2770
0: octal value of binary string uid,gid,sticky
1~3: octal value of binary string read,write,execute
chmod a[+-][rwx]
chmod [ug][+-][rwxs]
chmod u[+-]s         : uid
chmod g[+-]s         : gid
chmod [o]+-[rwxt]

find 101

find [{-H|-L|-P}] <path> <expression>

create iso image


for cdrom
umount /dev/cdrom
dd if=/dev/cdrom of=/tmp/file.iso


for general files
mkisofs -o /tmp/file.iso /path/to/directory/


read output of a command into multiple variables

read v1 v2 < <(echo val1 val2)

detect a remote host can be reached/sshed


use netcat(nc)
-z : scan only, no data sent

-w  : wait timeout

host=sh-debian8
port=22
nc -z -w 3 $host $port


use ssh
ssh -o PreferredAuthentications=publickey -o ConnectTimeout=3 $host


quick guide about iptables


simplified command syntax
iptables -t 
 -A/D/I/F/L <chain_of_table>  


tables with chains
filter
INPUT
OUTPUT
FORWARD
nat
PREROUTING
POSTROUTING
OUTPUT
mangle
PREROUTING
POSTROUTING
INPUT
OUTPUT
FORWARD


criteria determine what kind of packets


target determine where packets should go, can be ACCEPT/DROP/QUEUE/RETURN or a user chain


common tasks


add NAT setting
g='10.10.209.179'
hs='sh-focus1 sh-focus2'
for h in $hs; do iptables -t nat -A POSTROUTING -j SNAT -s $h -d host.company.com --to-source $g; done


add MASQUERADE
EXTERNAL_INTERFACE=eth0
iptables -t nat -A POSTROUTING -j MASQUERADE -o $EXTERNAL_INTERFACE


port forwarding
iptables -t nat -A PREROUTING -j DNAT -p tcp --dport 18888 --to 192.168.18.104:18888


redirect port
iptables -t nat -A PREROUTING -p tcp --dport 80 -j REDIRECT --to-port 18888


check packet by logging
iptables -A INPUT -p icmp -j LOG --log-prefix "[icmp input]"


how to leave command run in background via ssh

the command must be redirected
ssh 'command</dev/null &>/dev/null &'
refer to 'nohup' in wikipedia
code development in vim with plugins and ctags


plugin
cat ~/.vim/plugin/
minibufexpl.vim (buffer tab)
taglist.vim (source code structure)
vtreeexplorer.vim (tree-like explorer)
ctags
cat ~/.vimrc
set tags=tags;
set autochdir
ctags -R src/*
go tag: ctrl-]
back tag: ctrl-t
vim:
:ta  //go to tag
:ts       //tag list select

NFS

vi /etc/exports
/usr/sbin/exportfs

get date by offset

date -d "$(date +%Y-%m-15) -1 month"
date --date="$(date +%Y-%m-15) -1 month"

check if a user is a member of a netgroup

getent netgroup <netgroup> | grep <user>

Job in bash

"job -p" lists all background jobs
"wait " returns the exit value of background process
set pipe command exit status to the last failed one in the pipe

set -o pipefail

XML - how to mix attribute with sub-elements in an element in XSD

just define a complexType with mixed set to "true" and put attribute after
sequence of all elements
==========================================================
<xs:element name="condition" maxOccurs="unbounded">
<xs:complexType mixed="true">
xs:sequence
<xs:element name="type">
xs:simpleType
<xs:restriction base="xs:string">
<xs:enumeration value="What"/>
<xs:enumeration value="Where"/>
<xs:enumeration value="How"/>
<xs:enumeration value="Who"/>
<xs:enumeration value="When"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
<xs:element name="keyword" type="xs:string" minOccurs="1" maxOccurs="1"/>
<xs:element name="operator" type="xs:string" minOccurs="1" maxOccurs="1"/>
<xs:element name="value" type="xs:string" minOccurs="1" maxOccurs="1"/>
</xs:sequence>
<xs:attribute name="exception" type="xs:integer"/>
</xs:complexType>
</xs:element>
==========================================================
run local script remotely

ssh user@server 'bash -s' < script.sh

vsftpd config

# enable local user with upload
local_enable=YES
write_enable=YES
pam_service_name=vsftpd
# enable anonymous with write
anonymous_enable=YES
write_enable=YES
anon_upload_enable=YES
anon_other_write_enable=YES
hide_ids=YES

turn off selinux options


enable local home dir
setsebool -P ftp_home_dir on


enable anonymous write
setsebool -P allow_ftpd_anon_write on
setsebool -P allow_ftpd_full_access on


check selinux status
sestatus


get se bool settings
getsebool -a


set se bool
setsebool [-P] boolean value | bool1=val1 bool2=val2 ...


get/set enforce
getenforce
setenforce [ Enforcing | Permissive | 1 | 0 ]


check netgroup membership


linux
getent netgroup some-group | grep yangjian


solaris
niscat netgroup.org_dir|grep yangjian


install yum from local dvd

refer to /etc/yum.repos.d/CentOS-Media.repo
setup postfix mail server


add in DNS
      IN MX 10 smtp

lab.com IN MX 10 smtp.lab.com.
smtp    IN A xxx.xxx.xxx.xxx


install postfix


config /etc/postfix/main.cf
myhostname = smtp.lab.com
mydomain = lab.com
myorigin = $mydomain
inet_interfaces = all
mydestination = $mydomain, $myhostname, localhost.$mydomain, localhost


about gpg


sign
gpg --output doc.sig --sign doc


encrypt


decrypt
gpg --output doc.sig --decrypt doc.sig


verify
gpg --verify doc.sig


about cpio


cp set of files to archive
cp -ov > tree.cpio


cp set of files from archive,keep timestamp and create dir
cp -idmv < tree.cpio


cp set of files from dir to dir, keep timestamp and create dir
find . -depth -print0 | cpio --null -pdmv to_dir


cat a file list to cpio requires to remove \r
cat filelist | tr -d '\r' | cpio -pdmv to_dir


about join in SQL


join without join-predicate
# CROSS JOIN: Cartesian product, t1 X t2, no join-predicate, unconditionally
SELECT * FROM t1 CROSS JOIN t2;


join with join-predicate


result from matching record
#INNER JOIN: CROSS JOIN with join-predicate(like ON, USING)
SELECT * FROM t1 INNER JOIN t2 ON t1.x = t2.y;
SELECT * FROM t1 INNER JOIN t2 USING(x);
#EQUI-join: INNER JOIN with only equality comparision in join-predicate
#NATURAL JOIN: EQUI-JOIN by comparing all columns with same names
SELECT * FROM t1 NATURAL JOIN t2;


result from given table(LEFT/RIGHT/FULL)
#OUTER JOIN:
#LEFT OUTER JOIN/LEFT JOIN:
SELECT * FROM t1 LEFT JOIN t2 USING(x);
#RIGHT OUTER JOIN/RIGHT JOIN:
SELECT * FROM t1 RIGHT JOIN t2 USING(x);


tcpdump

# general syntax

tcpdump [options] {<type>|<dir>|<proto>} [{<type>|<dir>|<proto>} ...] <id>

# commonly used options
tcpdump -e : Print the link-level header(useful to print MAC address)
tcpdump -i <interface> : Listen on <interface>
tcpdump -l : Make stdout line buffered
tcpdump -n : Don't convert addresses
tcpdump -s <snaplen> : Snarf snaplen bytes of data from each packet rather than the default of 262144 bytes.
tcpdump -X : Print the data of packet in hex and ASCII, in addtion to the headers
tcpdump -XX: Print the data of packet in hex and ASCII, in addtion to the headers, including link level header

# examples

  # list all interfaces for packets capture
  sudo tcpdump -D
  
  # limit captured packets size
  sudo tcpdump -i any -s 96
  
  # check for DHCP packets from client(68) or server(67)
  sudo tcpdump -i eth0 -en -vv port 67 or port 68
  
  # use nmap to trigger DHCP request
  sudo nmap --script broadcast-dhcp-discover -e eth0

fonts in X


X11 bitmap
freetype(XFT)

subversion


create a repo

    # prepare svn account
    sudo useradd svn
    # prepare repo dir as root
    sudo mkdir /svn_repo
    chown svn:svn /svn_repo
    #login as svn
    svnadmin mkdir /svn_repo
    # create a project
    svn mkdir file:///svn_repo/project1/trunk
    svn mkdir file:///svn_repo/project1/branch
    svn mkdir file:///svn_repo/project1/tag


common usage

    # list branches
    svn list <repo_url>/project1
    # list commits on this branch
    svn log --stop-on-copy
    # list commits on a branch
    svn log --stop-on-copy <branch_url>
    # find branch root at parent branch:rev
    svn log -v  --stop-on-copy ^/CEC/Applications/CEPM/branch/$b \
       | grep -A1 'Changed paths' \
       | tail -1 | awk '{print $NF}' | sed 's=.*/==' -e 's/)//'
    # show ancestor branches of given $branch 
    branch=/CEC/Applications/CEPM/branch/cepm-3320-stable-HMP
    b=$branch;echo $b; \
    while expr $b : '.*/branch/'>/dev/null; do \
    b=$(svn log -v --stop-on-copy ^$b \
           | grep -A1 'Changed paths' \
           | tail -1|awk '{print $NF}'|sed 's/:.*//');\
    echo "<=$b"; \
    done
    # find out detail branch info for a revision under a URL
    svn log -v -c <revision> <url>
    # show diff ignore space
    svn diff --diff-cmd /usr/bin/diff -x -uw
    # revert last commit at revision xxxx
    svn merge -c -xxxx .
    # keep alive a reintegrated branch
    # assign xxxx=<reintegrated_revision_on_parent_branch>
    svn merge --record-only -c xxxx ^/project/parent
    svn commit -m 'blocking revision xxxx from being merged into this branch'
    # remove mime-type for text file mislabelled as binary file
    svn propdel svn:mime-type <file>


recover a removed branch

     # identify the revision previous to deleted one and copy it to branch
     svn copy <branch>@<revision_before_delete> <branch>

install vncserver in CentOS 6.2

sudo yum install tigervncserver libXfont libX11

enable Java in Firefox

# for 32bit Firefox
ln -sf <jdk>/jre/lib/i386/libnpjp2.so .mozilla/plugins/
# for 64bit Firefox
ln -sf <jdk>/jre/lib/amd64/libnpjp2.so .mozilla/plugins/

bash comment block

: << \#
comment line 1
comment line 2

splunk search

: sourcetype=cpdpa source=*prod*pdp.log "2013-12-08 19" 

Java regex tips

# use '\\' in case of any '\' due to Java because
Backslashes within string literals in Java source code are interpreted
as required by The Java? Language Specification as either Unicode
escapes (section 3.3) or other character escapes (section 3.10.6) It is
therefore necessary to double backslashes in string literals that
represent regular expressions to protect them from interpretation by the
Java bytecode compiler
#
* ignore case: "(?i)X"
* greedy: "X*"
* reluctant: "X*?"
* look-ahead(right):
    positive: "(?=X)"
    negative: "(?!X)"
* look-behind(left)
    positive: "(?<=X)"
    negative: "(?<!X)"

LDAP search

ldapsearch -h ldapserver -b "OU=xxys3cpi,OU=ConnectionFactory,OU=xxcepmsj.topic.scheduledjobs, OU=jmsObjects,OU=jms,OU=Apps,O=cco.company.com" -x "CN=ermo-test3"
ldapsearch -h ldapserver "CN=dev" -b "OU=Destination,OU=xxcpdpa.topic.policyupdate, OU=jmsObjects,OU=jms,OU=Apps,O=cco.company.com"
ldapsearch -h ldapserver "CN=dev" -b "OU=xxcpapa,OU=ConnectionFactory,OU=xxcpdpa.topic.policyupdate, OU=jmsObjects,OU=jms,OU=Apps,O=cco.company.com"
ldapsearch -h ldapserver "CN=dev" -b "OU=Destination,OU=xxcpapa.topic.reply, OU=jmsObjects,OU=jms,OU=Apps,O=cco.company.com"
ldapsearch -h ldapserver "CN=dev" -b "OU=xxcpdpa,OU=ConnectionFactory,OU=xxcpapa.topic.reply, OU=jmsObjects,OU=jms,OU=Apps,O=cco.company.com"
ldapsearch -h ldapserver \
    -x \
    -D 'uid=ecepmdsx.gen,OU=Generics,O=cco.company.com' \
    -w 'cepm$cco' \
    -b "ou=ccoentities,o=cco.company.com" \
    "uid=yangjian" \
    uid \
    manager \
    accesslevel
# search by manager has to use full string equal
ldapsearch -h dsx \
    -x \
    -b 'ou=ccoentities,o=cco.company.com' \
    'manager=uid=changche,OU=ccoentities,O=cco.company.com' \
    uid accessLevel employeeType manager

# ldapsearch inactive users
ldapsearch -h dsx \
    -x \
    -D 'uid=ecepmdsx.gen,OU=Generics,O=cco.company.com' \
    -w 'cepm$cco' \
    -b 'ou=inactive,o=cco.company.com' \
    -Epr=3/prompt \
    uid accessLevel employeeType manager

# ldapsearch with paged result
ldapsearch -h dsxstage.company.com \
    -x \
    -b 'ou=ccoentities,o=cco.company.com' \
    -Epr=3/prompt \
    'manager=uid=changche,OU=ccoentities,O=cco.company.com' \
    uid

# check for support of Persistent Search(1.2.840.113556.1.4.528) in AD
ldapsearch -h dsxstage.company.com -x -s base -b '' 'objectclass=*' supportedControl | grep 528

# search with TLS
LDAPTLS_REQCERT=never \
ldapsearch  \
    -ZZ \
    -h ds.company.com \
    -x \
    -D 'CN=coi-kongfig.gen,OU=Generics,OU=Company Users,DC=company,DC=com' \
    -w 'GB5vesBt' \
    -b 'OU=Generics,OU=Company Users,DC=company,DC=com' \
    cn=coi-kongfig.gen \
    dn cn memberOf

variables in SQLPlus


PLSQL Variables
declaration
definition: vname := 'value';
usage: vname
SQLPlus variables

Substitution Variables
declaration&definition: DEFINE vname = 'value';
usage: &vname
Bind Variables
declaration: VARIABLE vname VARCHAR2;
definition: :vname := 'value';
usage: :vname


sqlplus to connect Oracle DB

user=cepm
pass=pass
url='(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=host1.company.com)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=SVCNAME1.company.com)))'
sqlplus $user/$pass@$url

Oracle control files for local use


~/.tnsnames.ora : local TNS names for Oracle
~/.sqlnet.ora : local profile configuration for Oracle

connect to Oracle database(e.g. CEPM@CEPMSTG) with wallet in sqlplus


lookup TNS string by tnsping
$ str=ssh -q ccixuser-nprd3-03 tnsping CEPMSTG | sed -n 's/Attempting to contact //p'


add database name to TNS string mapping in ~/.tnsnames.ora
echo "CEPMSTG=$str" >> ~/.tnsnames.ora


create a wallet
$ mkstore -wrl ~/ -create


create credential in wallet
$ mkstore -wrl ~/ -createCredential CEPMSTG CEPM
Enter your secret/Password:********
Re-enter your secret/Password:********
Enter wallet password:********


connect with sqlplus
$ sqlplus /@CEPMSTG


cleanup undo tables space in Oracle

create undo tablespace undotbs2 datafile '/tmp/undotbs02.dbf' size 1G autoextend off;
alter system set UNDO_TABLESPACE=undotbs2;
drop tablespace undotbs1 including contents and datafiles;

search for string in package body of all packages

select distinct name
from user_source
where 1=1
and type='PACKAGE BODY'
and lower(text) like '%iam_subject_dfltpg_mapping%'
;

bc function

scale=4
define pmt(p,r,n) { return r*p/(1-(1+r)^(-n)); }

XML file head must exist for a HTTP body to be recognized as XML format

<?xml version="1.0" encoding="UTF-8"?>

Git file operation

      ================== add ==============>
                               ==== add ===>
                                             ==== commit ======>
                                             = commit --amend =>
                               <= reset HEAD =
                   <= checkout -- =>

UNTRACKED <=> UNMODIFIED <=> MODIFIED <=> STAGED    <=>    COMMITTED

use csplit to split multi-cert file into separate ones by BEGIN line


first cert starts at cert1.crt, cert0.crt is empty
csplit -f cert -b %d.crt certs '/BEGIN/' '{*}' ; rm -f cert0.crt


first cert starts at cert0.crt
csplit -f cert -b %d.crt certs '%BEGIN%' '/BEGIN/' '{*}'


log to syslog in shell


log to /var/log/messages only
logger some message


log to /var/log/messages and stdout
logger -s some message


tmux sharing


create a new session and share it within group
tmux -S /tmp/yj/shareds new -s shared
chgrp eng /tmp/yj/shareds


access shared session
tmux -S /tmp/yj/shareds -t shared


view-only
tmux -S /tmp/yj/shareds -t shared -r


git submodule

* add
* remove
git submodule deinit -f <module_path>
git rm -f <module_path>
rm -rf .git/modules/<module_path>

find a set of files and package in a tar ball

# use find -print0 with tar -T
find dir -print0 | tar -czf t.tar.gz -T -

send email by SMTP with mail

echo body|mail -S smtp:outbound.company.com:25 -s subject

put text into clipboard


Linux
echo 'text to copy' | xclip


Mac
echo 'text to copy' | pbcopy


find files modified since 

# generic one
p='30 days ago' ; f=/tmp/yj/k; touch -t $(date +%y%m%d%H%M -d "$p") $f; find . -newer $f
# new version of find
find dir -newermt $(date -d '20161101')
# or simply
find . -type d -newermt 20220625  ! -newermt 20220626

Git


show diff of a commit
git show 


show changeset of a commit
git show --name-only 


at


list jobs
atq


rm job 3
atrm 3


schedule a notification at 12:58 Feb 23
echo "DISPLAY=$DISPLAY notify-send title body" | at 12:58 Feb 23


Elasticsearch

Mappings

Add "fields" definition to a field, to enable multiple types of same field for different usage.
Refer to multi-fields in ES guide