Skip to content

Instantly share code, notes, and snippets.

@liangfu
liangfu / ffmpeg.md
Created May 24, 2017 04:46 — forked from protrolium/ffmpeg.md
using ffmpeg to extract audio from video files

ffmpeg

Converting Audio into Different Formats / Sample Rates

Minimal example: transcode from MP3 to WMA:
ffmpeg -i input.mp3 output.wma

You can get the list of supported formats with:
ffmpeg -formats

Convert WAV to MP3, mix down to mono (use 1 audio channel), set bit rate to 64 kbps and sample rate to 22050 Hz:

@liangfu
liangfu / mldev_init.sh
Last active July 18, 2023 17:24
Initialize dev workspace with a single bash script
apt update
apt install -y emacs-nox aptitude screen xcscope-el gdb nano || echo 0 && echo "Success"
python3 -m pip install git-remote-codecommit opennmt-tf==1.25.1
cat << EOF > ~/.gdbinit
set print thread-events off
define hook-run
catch throw
end
define hook-quit
import numpy as np
import time
import math
def burn_cpu():
a = np.random.rand(1024, 1024)
b = np.random.rand(1024, 1024)
tic = time.time()
c = np.dot(a, b)
toc = time.time()
@liangfu
liangfu / s3_controller.py
Created January 28, 2021 09:15
Enable batch operations for recursively copying s3 objects
import boto3
def recursive_copy(src_bucket, src_prefix, dst_bucket, dst_prefix):
def get_partition():
if boto3.session.Session().region_name in ["cn-northwest-1", "cn-north-1"]:
return "aws-cn"
else:
return "aws"
partition = get_partition()
region_name = boto3.session.Session().region_name
@liangfu
liangfu / mldev.sh
Created January 30, 2023 21:24
Manage AWS EC2 instances with a single-line command
#!/bin/sh
ProgName=$(basename $0)
instance_ids=i-05806fb95d65fffff
region=us-west-2
sub_help(){
echo "Usage: $ProgName <subcommand> [options]\n"
echo "Subcommands:"
echo " start Start mldev instances"
@liangfu
liangfu / example_dev_tabular.py
Last active January 12, 2023 18:54
Benchmark inference time of pretrained AutoGluon Tabular models
# Benchmark inference time of pretrained AutoGluon Tabular models
"""
set -e
DEV_BRANCH=reset-thread-2
git checkout master
/opt/conda/envs/py38/bin/python3 example_dev_tabular.py --sample=1
/opt/conda/envs/py38/bin/python3 example_dev_tabular.py
git checkout $DEV_BRANCH
/opt/conda/envs/py38/bin/python3 example_dev_tabular.py --sample=1
/opt/conda/envs/py38/bin/python3 example_dev_tabular.py
""" Example script for defining and using custom models in AutoGluon Tabular """
from autogluon.core.utils import infer_problem_type
from autogluon.tabular import TabularDataset, TabularPredictor
from autogluon.tabular.configs.hyperparameter_configs import get_hyperparameter_config
from autogluon.core.data import LabelCleaner
from autogluon.core.models import AbstractModel
from skl2onnx import convert_sklearn, get_model_alias
from skl2onnx.common._registration import get_shape_calculator, get_converter
@liangfu
liangfu / bfloat16.cc
Last active January 4, 2022 23:00
An implementation of IEEE 16-bit floating point data type
#include <stdio.h>
#include <inttypes.h>
static const uint32_t kSingleSignMask = 0x80000000;
static const uint32_t kSingleExpMask = 0x7f800000;
static const uint32_t kSingleMantMask = 0x007fffff;
static const uint32_t kHalfSignMask = 0x8000;
static const uint32_t kHalfExpMask = 0x7c00;
static const uint32_t kHalfMantMask = 0x03ff;
# Inspired by https://keon.io/deep-q-learning/
# python3 -m pip install keras==2.3.1 tensorflow-gpu==1.15 gym==0.18.3
# python3 -m pip install 'h5py==2.10.0' --force-reinstall
import random
import gym
import math
import numpy as np
import keras
@liangfu
liangfu / bosonnlp_to_bio2.py
Last active January 27, 2021 07:13
Convert origindata.txt in bosonnlp to CoNLL format for named entity recognition (NER).
import random
random.seed(111)
def bosonnlp_to_bio2(origfile, trainfile, valfile):
val_ratio = 0.2
traindata = []
valdata = []
with open(origfile, 'rt') as fp: