Skip to content

Instantly share code, notes, and snippets.

Avatar

Son Mai sonhmai

  • Ho Chi Minh city, Vietnam
View GitHub Profile
View pg.sh
psql -U postgres
postgres=# SHOW config_file;
View xgb.py
# XGBClassifier
param_dist = {'objective': 'binary:logistic', 'n_estimators': 2}
clf = xgb.XGBClassifier(**params_dist)
View pyspark.sql.DataFrame.py
class pyspark.sql.DataFrame
people = spark.read.parquet("...")
department = spark.read.parquet("...")
people.filter(people.age > 30) \
.join(department, people.deptId == department.id) \
.groupBy(department.name, "gender") \
.agg({"salary": "avg", "age": "max"})
View git.sh
# saves credentials
$ git config credential.helper store
$ git push http://example.com/repo.git
Username: <type your username>
Password: <type your password>
@sonhmai
sonhmai / install-python.sh
Created Aug 24, 2018
Install Python 3.6.5 in Ubuntu 16+
View install-python.sh
#!/bin/bash
export PYTHON_VERSION=3.6.5
export PYTHON_DOWNLOAD_URL=https://www.python.org/ftp/python/$PYTHON_VERSION/Python-$PYTHON_VERSION.tgz
sudo apt update
sudo apt install --no-install-recommends -y \
software-properties-common build-essential \
libssl-dev libreadline-dev libbz2-dev libsqlite3-dev zlib1g-dev \
python-minimal
@sonhmai
sonhmai / list.rb
Created Aug 24, 2018 — forked from thbkrkr/list.rb
Kafka JMX MBeans list
View list.rb
com.sun.management:type=DiagnosticCommand
com.sun.management:type=HotSpotDiagnostic
java.lang:name=CodeCacheManager,type=MemoryManager
java.lang:name=Code Cache,type=MemoryPool
java.lang:name=Compressed Class Space,type=MemoryPool
java.lang:name=G1 Eden Space,type=MemoryPool
java.lang:name=G1 Old Generation,type=GarbageCollector
java.lang:name=G1 Old Gen,type=MemoryPool
java.lang:name=G1 Survivor Space,type=MemoryPool
java.lang:name=G1 Young Generation,type=GarbageCollector
@sonhmai
sonhmai / kafka
Last active Apr 29, 2021
kafka basic commands
View kafka
# BENCHMARK-----------------------------------------------------
#1. Rust kafka-benchmark (https://github.com/fede1024/kafka-benchmark)
# must create topic with 6 partitions first
bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 6--topic [scenario]
# replace scenario with one in file kafka-benchmark/config/base_producer.yaml
View basic-psysopg2.py
#basic python code for working with POSGRESQL through psycopg2
import psycopg2
conn = psycopg2.connect(database="testdb", user="postgres", password="pass123", host="127.0.0.1", port="5432")
print("Opened database sucessfully")
cur = conn.cursor()