Python and Unix Commands

memory.py

# Option 1
import os
from pathlib import Path
filename = Path(__file__).parts[-1]
mem = os.popen("ps aux | grep %s | awk '{sum=sum+$6}; END {print sum/1024 \" MB\"}'"% (filename)).read()

# Option 2
import os
import psutil # pip install psutil 
import humanize # pip install humanize
process = psutil.Process(os.getpid())
mem = humanize.naturalsize(process.memory_info().rss)

# Option 3 - GPU Memory
import pynvml # pip install pynvml3
pynvml.nvmlInit()
device_id = pynvml.nvmlDeviceGetHandleByIndex(0)
mem = pynvml.nvmlDeviceGetMemoryInfo(device_id).used/1024.0/1024.0/1024.0 # (GB)

process.md

Using top command

• To check processes by a user
  • top -u <username>
• To check for process(es)
  • top -p <pid1>,<pid2>
• To cycle through Kb/Mb/Gb ...
  • Capital E - for system level info
  • Small e - for process level info
• To check mem usage by code and data
  • Small f - and then space to add or remove a column
• By default this shows the process id (pid) of a process
  • hitting "H" also shows the threads
  • hitting "U" and then typing your username allows to see only your users pids (in a cluster scenario)

Using the ps command

• To see which processor a pid is running on
  • watch -n 0.1 'ps -o pid,psr,comm -p <pid>' (dunno whats it practical use case)
• To count the thread a process has spawned
  • ps -o thcount <pid> (thcount=thread count)
  • ps -o nlwp <pid> (nlwp=number of light weight processes)
  • cat /proc/<pid>/status| grep Threads
• Getting memory (in MB)
  • ps -o rss -p <pid> | awk 'NR>1 {$0=($0/1024);}{print;}'

remove_dir.py

from pathlib import Path

def rm_tree(path):
    for child in path.iterdir():
        if child.is_file():
            child.unlink()
        else:
            rm_tree(child)
    path.rmdir()
 
rm_tree(Path(./)) # be careful!!

terminal.md

1. Disk Usage
   • du -shc ./*
2. Searching for files
   • Basic searching
     • find ./ -name *.code-workspace
   • Finding the latest upated files
     • find ./ -type f -mtime -1 -ls
   • With tree command
     • sudo apt install tree
     • tree --prune -P '{regex}' {search-directory}