Skip to content

Instantly share code, notes, and snippets.

Avatar

Junjue Wang junjuew

View GitHub Profile
@junjuew
junjuew / phash_dedup.py
Last active Nov 3, 2020
Deduplicate Images Based on Their Perceptual Hash
View phash_dedup.py
import pathlib
import cv2 as cv
import fire
import imagehash
from PIL import Image
def dedup(video_uri: str, output_dir: str, min_hash_difference_between_key_frames: float=10):
cap = cv.VideoCapture(video_uri)
@junjuew
junjuew / cpu-freq-performance.sh
Created May 23, 2020
set CPU frequency to performance on linux
View cpu-freq-performance.sh
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
# https://wiki.archlinux.org/index.php/CPU_frequency_scaling
@junjuew
junjuew / test.py
Last active Mar 2, 2021
Set abseil / Tensorflow flags from python script
View test.py
"""Example for passing Tensorflow configuration variables from python script.
Many tensorflow scripts variable are defined with absl, which parses inputs from
command line by default. Here is a way to set the flags in a python script.
"""
from absl import flags
FLAGS = flags.FLAGS
flags.DEFINE_integer('my_version', 0, 'Version number.')
@junjuew
junjuew / gist:b0578e9b486df7461f40435ddbc0c787
Created Nov 18, 2019
Python One-Liner to Show a Single Example from TFRecord
View gist:b0578e9b486df7461f40435ddbc0c787
python -c 'import sys; import tensorflow as tf; ex=next(tf.python_io.tf_record_iterator(sys.argv[1])); print(tf.train.Example.FromString(ex))' <path-to-tfrecord>
@junjuew
junjuew / docker-firewall.sh
Last active Aug 19, 2019
Set up firewalls for docker container to allow internal/CMU IPs only
View docker-firewall.sh
# in Docker >v17, DOCKER-USER chain is introduced. It is a
# iptable chain in FILTER table, called before DOCKER chain.
# By default, it accepts all connections.
# delete default return rule
sudo iptables -D DOCKER-USER -j RETURN
# allow default container and private subnet
sudo iptables -C DOCKER-USER -s 172.16.0.0/12 -j ACCEPT || sudo iptables -I DOCKER-USER 1 -s 172.16.0.0/12 -j ACCEPT
sudo iptables -C DOCKER-USER -s 192.168.0.0/16 -j ACCEPT || sudo iptables -I DOCKER-USER 1 -s 192.168.0.0/16 -j ACCEPT
sudo iptables -C DOCKER-USER -s 10.0.0.0/8 -j ACCEPT || sudo iptables -I DOCKER-USER 1 -s 10.0.0.0/8 -j ACCEPT
# allow CMU ips
@junjuew
junjuew / tensorflow-object-detection-benchmark.py
Last active Feb 16, 2020
tensorflow object detection inference speed benchmark
View tensorflow-object-detection-benchmark.py
import os
import tarfile
import time
import numpy as np
import six.moves.urllib as urllib
import tensorflow as tf
tf.logging.set_verbosity(tf.logging.INFO)
@junjuew
junjuew / uninstall_cloud_init_add_passwd.sh
Created Jul 10, 2018
Add password login to cloud images
View uninstall_cloud_init_add_passwd.sh
virt-customize -a /var/lib/libvirt/images/debian9-vm1.qcow2 --root-password password:NewRootUserPasswordHere --uninstall cloud-init
@junjuew
junjuew / vm.xml
Last active Jun 27, 2018
libvirt xml with port forwarding
View vm.xml
# Use qemu directly with kvm
# sudo qemu-system-i386 -machine accel=kvm -m 2048 mn-trusty64server-170321-14-17-08/mininet-vm-x86_64.vmdk -net nic,model=virtio -net user,net=192.168.101.0/24,hostfwd=tcp::8022-:22
# sample libvirt xml with port forwarding enabled
# copy the following
<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
<name>mininet</name>
<uuid>cd4dbb45-1628-4310-9a1a-f8bc62ca91c0</uuid>
<memory unit='KiB'>4194304</memory>
<currentMemory unit='KiB'>4194304</currentMemory>
<vcpu placement='static'>2</vcpu>
@junjuew
junjuew / keybindings.json
Created Jun 21, 2018
Custom Visual Studio Code Keybinding
View keybindings.json
// Place your key bindings in this file to overwrite the defaults
// File path on Ubuntu is $HOME/.config/Code/User/keybindings.json
[
// ctrl+shift to navigate among UI elements
{
"key": "ctrl+shift+up",
"command": "-workbench.action.terminal.scrollUp",
"when": "terminalFocus"
},
@junjuew
junjuew / license.py
Created May 6, 2018
A bare-minimal script to add and remove licenses on top of python files.
View license.py
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import fire
LICENSE_CONTENT = {}
LICENSE_CONTENT['apache'] = """# Copyright (C) 2018 Carnegie Mellon University. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");