Skip to content

Instantly share code, notes, and snippets.

View refraction-ray's full-sized avatar

Shixin Zhang refraction-ray

View GitHub Profile
@refraction-ray
refraction-ray / tcdns.sh
Last active January 9, 2018 04:18
a CLI utility tool to examine DNS availability
#! /bin/bash
while getopts "46thvs:" arg
do
case $arg in
s)
sflag=x
dnsip=$OPTARG
;;
4)
v4flag=x
@refraction-ray
refraction-ray / f-index.py
Created January 11, 2018 13:48
python3 script designed for Alfred3 workflow, view multiple finance index at the same time
from twisted.internet import reactor
import scrapy
from scrapy.crawler import CrawlerRunner
from scrapy.utils.log import configure_logging
import json
import re
from scrapy.downloadermiddlewares.useragent import UserAgentMiddleware
result=[]
@refraction-ray
refraction-ray / checkdomains.py
Last active January 12, 2018 15:40
CLI tool to confirm whether there are two black list or just one
from re import sub,search
from subprocess import check_output
with open("GFWlist.txt") as ls:
total = 0
rst = 0
poison = 0
for line in ls.readlines():
domain = sub(r".*--.*|^@@.*|.*##.*|^\[.*|!.*|http://|https://|[\s]*|^\^https.*","",line)
domain = sub(r"^\.(.*)$|^[|]{1,2}(.*)$","\\1",domain)
if bool(search(r"^[_a-zA-Z0-9-]+(\.[_a-zA-Z0-9-]+)*\.[a-z]+$",domain)):
@refraction-ray
refraction-ray / ipoption.py
Last active May 2, 2022 16:44
extension functions on IPOption settings in scapy
from scapy.all import sr1, IP, IPOption,ICMP
import ipaddress as ipa
def record_route(hop=1):
if hop >= 10:
raise Exception('exceed the max length of ip header')
length = chr(hop*4+3)
option_head = b'\x07'+length.encode()+b'\x04'
route_data = "".join(['\x00']*4*hop)
option = option_head+('%s'%route_data).encode()
@refraction-ray
refraction-ray / htmldemo.ipynb
Last active February 13, 2018 00:36
a demo on how the jupyter notebooks show html content
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@refraction-ray
refraction-ray / jq_shell_demo.ipynb
Last active February 11, 2018 12:19
demo on jupyter notebook running shell commands in Joinquant
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@refraction-ray
refraction-ray / index_pepb.py
Created April 26, 2018 02:14
joinquant code snippet to visualize on pe,pb history rank for several index
import pandas as pd
import datetime as dt
import os
import numpy as np
import jqdata
import bisect
import matplotlib.pyplot as plt
from pyecharts import Page,online,Line,Grid
import warnings
warnings.filterwarnings('ignore')
@refraction-ray
refraction-ray / demo_nlp_arxiv.ipynb
Created December 7, 2018 09:42
demo_NLP_arxiv.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@refraction-ray
refraction-ray / README.md
Last active December 11, 2018 09:51
The patch file for apsrev.bst

Even the official .bst file provided by APS in revtex4.1 is not satisfying in terms of the format of citations, including the redundant inclusion of eprint no. for published works, and super long book titles or conference titles without newlines.

Therefore, I provide a patch file here as a slight change of the original style. This patch file can be applied to apsrev.bst, which can be found here.

patch apsrev.bst apsrev.patch
@refraction-ray
refraction-ray / README.md
Last active January 3, 2019 05:13
Python import mechanism

This is a demo package to show the implementation details of python import system. Collect the above py files into a folder app. The folder orginization is

root
├── ext.py
│   
├── app
│   ├── __init__.py
│   ├── a.py