Skip to content

Instantly share code, notes, and snippets.


Herminio Vazquez canimus

View GitHub Profile
canimus /
Last active May 9, 2021
Pyspark Cast 100 columns
from functools import reduce, methodcaller
import pyspark.sql.functions as F
_ts = lambda dataFrame, col: methodcaller('withColumn', f'{col}TimeStamp', F.to_timestamp(F.col(col)/1000))(dataFrame)
reduce(lambda a,b: _ts(a,b), ['start','stop'], df).select('startTimeStamp', 'stopTimeStamp').show(truncate=False)
canimus / AWS Swarm
Created Mar 29, 2021 — forked from ghoranyi/AWS Swarm
Create a Docker 1.12 Swarm cluster on AWS
View AWS Swarm

This gist will drive you through creating a Docker 1.12 Swarm cluster (with Swarm mode) on AWS infrastructure.


You need a few things already prepared in order to get started. You need at least Docker 1.12 set up. I was using the stable version of Docker for mac for preparing this guide.

$ docker --version
Docker version 1.12.0, build 8eab29e

You also need Docker machine installed.

canimus /
Created Mar 16, 2021
IBM Web Server Log Parse
from hashlib import md5 as xx
from collections import namedtuple
import os
import re
# Environment parametes
COLUMN_SEPARATOR = chr(os.getenv('SEPARATOR', 449))
FILE_NAME = os.getenv('FILE', 'hashed.csv')
# Regular expression to capture JSESSIONID
canimus /
Created Feb 15, 2021
Pie Chart Matplotlib
import matplotlib.pyplot as plt
# create data
names='Failed: 20', 'Passed: 80',
size=[11.8, 98.2]
# Create a circle for the center of the plot
my_circle=plt.Circle( (0,0), 0.7, color='white')
plt.pie(size, labels=names, colors=['#f56262', '#23d993'])
find . -type f -exec git log --reverse --format="{} %cn" -1 {} \; | cut -d" " -f2- | sort | uniq -c | sort -n -k1
rg ".*I wait for (\d+) secon.*$" -N -r '$1' | rg "\d+" | awk -F: '{s[$1]+=$2}END{for (i in s) print i,s[i]}' | sort -n -k2 | awk '{s+=$2}END{print s}'
canimus / tag_count.js
Created Feb 1, 2021
Count tags in web page with webdriverio
View tag_count.js
() => {
return Array.from(document.querySelectorAll("*")).map(e=>e.tagName.toLowerCase())
.reduce((a,b) => {a[b]=(a[b] || 0)+1; return a}, {})
canimus /
Created Jan 7, 2021
termgraph test cases
import glob
import json
import chalk
import asyncio
import aiofiles
import tabulate
import pandas as pd
import itertools as it
canimus /
Created Jan 4, 2021
PySpark Union JSON
df = reduce(lambda a,b: a.union(b), ['{x}/*result.json') for x in range(195, 202)])
canimus / testcase.ipynb
Last active Oct 28, 2020
Test Case - Rules
View testcase.ipynb
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.