Skip to content

Instantly share code, notes, and snippets.

💭
workon on https://github.com/omegaml

miraculixx

💭
workon on https://github.com/omegaml
  • Switzerland
Block or report user

Report or block miraculixx

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@miraculixx
miraculixx / rules.py
Last active Sep 3, 2019
a simple python based rule engine
View rules.py
"""
(c) 2014 miraculixx at gmx.ch
"""
from shrutil.dictobj import DictObject
class RuleContext(DictObject):
"""
rule context to store values and attributes (or any object)
"""
def __init__(self):
@miraculixx
miraculixx / .bash_k8s.rc
Created Jul 25, 2019
useful kubectl shortcuts
View .bash_k8s.rc
# put in $HOME/.bash_k8s.rc
# . $HOME/bash_k8s.rc
# podlog <any pod name substring>
function podlog() {
kubectl get pods -o name | grep $1 | cut -d / -f 2 | head -n1 | xargs kubectl logs $2
}
# exec a command in a pod, defaults to bash
function podssh() {
View how-to-install-latest-gcc-on-linux-mint.txt
These commands are based on a askubuntu answer http://askubuntu.com/a/581497 and https://askubuntu.com/questions/26498/choose-gcc-and-g-version
To install gcc-7 (gcc-7.2.0), I had to do more stuff as shown below.
USE THOSE COMMANDS AT YOUR OWN RISK. I SHALL NOT BE RESPONSIBLE FOR ANYTHING.
ABSOLUTELY NO WARRANTY.
If you are still reading let's carry on with the code.
sudo apt-get update && \
sudo apt-get install build-essential software-properties-common -y && \
sudo add-apt-repository ppa:ubuntu-toolchain-r/test -y && \
@miraculixx
miraculixx / requirements.txt
Created Nov 28, 2018
Python multiprocess parallel selenium web scraping with improved performance
View requirements.txt
beautifulsoup4==4.6.3
certifi==2018.10.15
chardet==3.0.4
idna==2.7
lxml==4.2.5
requests==2.20.1
selenium==3.141.0
urllib3==1.24.1
View gitdeploy
#!/usr/bin/env bash
# adopted from https://raw.githubusercontent.com/X1011/git-directory-deploy/master/deploy.sh
#
# changes
# * deploy from current directory is possible
# * set options from either .deploy file or parameters
set -o errexit #abort if any command fails
PREFIX=.
CONFIG=.deploy
@miraculixx
miraculixx / readcsv.py
Created Mar 1, 2019
read a large csv file and subset on datetime column
View readcsv.py
from dateutil.parser import parse
with open('data.csv') as fin:
for line in csv.reader(fin):
date1 = parse(row[1])
date2 = datetime.now() - timedelta(days=3)
if date1 > date2:
continue
process(row) # whatever your logic
@miraculixx
miraculixx / sample.py
Last active Feb 27, 2019
Read arbitrary formatted file into pandas dataframe
View sample.py
import pandas as pd
text = """
Men super men size Energy (J) type num g
----------------------------------------------------------------------
50 1 1 1.0234E+03 A abcd 12.1
20 7 4 5.0211E+02 A2 C agcd 14.1
10 2 3 -1.0347E+02 B2 abkd 72.1
"""
@miraculixx
miraculixx / socratapager.py
Created Feb 22, 2019
sodapy.Socrata extenion to page through results using a single call to get
View socratapager.py
from sodapy import Socrata
class SocrataPager(Socrata):
def get(self, dataset, *args, limit=None, offset=0, where=None, order=None, page_size=None, **kwargs):
""" get dataset by pages or within limit
uses Socrata.get to page through results until limit is reached. This adds the page_size parameter
which is passed as the limit kwarg to Socrata.get. limit is the total number of records to
return, across all pages. Note that the method is a generator.
@miraculixx
miraculixx / github-add-user
Last active Jan 28, 2019
batch adding users to github accounts
View github-add-user
#!/bin/bash
# Collaborator list, add, remove from a repository
# (c) 2015 miraculixx
# Author: github.com/miraculixx
# MIT License, see below
function help {
echo "Add collaborators to one or more repositories on github"
echo ""
echo "Syntax: $0 -u user -p password [-l] [-D] -r repo1,repo2 <collaborator id>"
@miraculixx
miraculixx / cors.py
Last active Jan 12, 2019
To enable CORS support in django-tastypie, use the following code snipet. Then create your tastypie resources as a subclass of BaseCorsResource.Basic code courtesy Daniel Conzalez of http://codeispoetry.me/index.php/make-your-django-tastypie-api-cross-domain/.I added documentation and the post_list method.
View cors.py
'''
Add CORS headers for tastypie APIs
Usage:
class MyModelResource(CORSModelResource):
...
class MyResource(CORSResource):
...
You can’t perform that action at this time.