adamwight

amsa@C235:~/editquality$ python differ.py "Japanese Wikipedia"
  
- ########################### Japanese Wikipedia ################################
+ ############################# Japanese Wikipedia ################################
? ++
- 
  # From https://quarry.wmflabs.org/query/9927
  datasets/jawiki.sampled_revisions.40k_2016.json:
  	wget -qO- https://quarry.wmflabs.org/run/89016/output/0/json-lines?download=true > $@
  

adamwight

amsa@C235:~/editquality$ python differ.py "Japanese Wikipedia"
  
- ########################### Japanese Wikipedia ################################
+ ############################# Japanese Wikipedia ################################
? ++
- 
  # From https://quarry.wmflabs.org/query/9927
  datasets/jawiki.sampled_revisions.40k_2016.json:
  	wget -qO- https://quarry.wmflabs.org/run/89016/output/0/json-lines?download=true > $@
  

adamwight

Step 0: Set up python virtualenv

virtualenv is a command-line utiltity that will allow you to encapsulate a python environment. Ubuntu calls the package that installs this utility "python-virtualenv". You can install it with $ sudo apt-get install python-virtualenv.

Step 1: Create the virtualenv directory

$ cd ~
$ mkdir venv
$ virtualenv -p python3 --system-site-packages ~/venv

adamwight

memuse measures unique physical total memory taken by a process and its children, ignoring duplicate copy-on-write pages and shared memory.

This is a solution for http://serverfault.com/questions/676335/how-measure-memory-without-copy-on-write-pages

It's a quick and dirty utility, but feel free to fork & improve.

Example:

~ » sudo ./memuse.py 15897                                                                                                                         eugene@eugene-thinkpad

PID Commandline Frames (+unique) VMEM