Skip to content

Instantly share code, notes, and snippets.

View c0mpiler's full-sized avatar

c0mpiler c0mpiler

View GitHub Profile
@c0mpiler
c0mpiler / test1.py
Created March 22, 2023 19:34
import sys
from operator import add
from pyspark.sql import SparkSession
if __name__ == "__main__":
spark = SparkSession\
.builder\
.appName("PythonWordCount")\
@c0mpiler
c0mpiler / submit.sh
Created March 22, 2023 19:33
#! /usr/bin/env bash
time spark-submit \
--deploy-mode client \
--master spark://spark-master:7077 \
--conf spark.driver.host=spark-master \
--conf spark.executor.memory=2g \
--conf spark.driver.memory=1g \
"$@"
@c0mpiler
c0mpiler / test.js
Created August 30, 2020 14:37
test.js
console.log("testing")
@c0mpiler
c0mpiler / youtube-pihole-adblock.sh
Created July 28, 2020 10:55
youtube-pihole-adblock.sh
echo "created for public use and use."
echo "Start with re-install cleanup."
rm -r /etc/dnsdumpster
rm /var/www/html/youtube-ads-list.txt
rm /etc/pihole/youtube-ads.sh
echo "cleanup done."
echo "installing python-pip and dnsdumpster."
apt-get install python-pip
@c0mpiler
c0mpiler / resource-visualisation-cartography.md
Created July 25, 2020 15:22 — forked from dmc-at-work/resource-visualisation-cartography.md
Resource Visualisation - Cartography

Resource Visualisation - Cartography

Via VM, Local Machine or Vagrant

Prerequisites

Operating System: Ubuntu 18.04

Prepare the following information

AWS Region (e.g. ap-southeast-1)
@c0mpiler
c0mpiler / logo_ascii.txt
Last active April 17, 2019 23:59
logo_ascii.txt
*****
+**************
********************
************************
****************************
.*******************************
************** ****************
*********** *************
********** : ***********.
@c0mpiler
c0mpiler / omxware_gene_2943ed16fb0889ef5e372113d97f56d0.json
Created April 16, 2019 18:11
omxware_gene_2943ed16fb0889ef5e372113d97f56d0.json
{
"@timestamp": "2019-04-16T06:21:07.417Z",
"@version": "1",
"omxid": "2943ed16fb0889ef5e372113d97f56d0",
"name": [
"Elongation factor Tu 1"
],
"type": "gene",
"sequence": "ATGTCTAAAGAAAAGTTTGAACGTACAAAACCGCACGTTAACGTCGGTACTATCGGCCACGTTGACCATGGTAAAACAACGCTGACCGCTGCCATCACTACCGTACTGGCTAAAACCTACGGTGGTTCCGCTCGCGCATTCGACCAGATCGATAACGCGCCGGAAGAAAAAGCTCGTGGTATCACCATCAACACCTCTCACGTTGAATATGACACCCCGACTCGCCACTACGCGCACGTAGACTGCCCGGGCCACGCCGACTATGTTAAAAACATGATCACCGGTGCTGCGCAGATGGACGGCGCGATCCTGGTTGTTGCTGCGACTGACGGCCCGATGCCGCAGACTCGTGAGCACATCCTGCTGGGTCGTCAGGTAGGCGTTCCGTACATCATCGTGTTCCTGAACAAATGCGACATGGTTGATGACGAAGAGCTGCTGGAACTGGTTGAGATGGAAGTTCGTGAACTGCTGTCTCAGTACGATTTCCCGGGCGACGACACTCCGATCGTTCGTGGTTCTGCTCTGAAAGCGCTGGAAGGCGACGCAGAGTGGGAAGCGAAAATCATCGAACTGGCTGGCCACCTGGATACCTATATCCCGGAACCAGAGCGTGCGATTGACAAGCCGTTCCTGCTGCCGATCGAAGACGTATTCTCCATCTCCGGTCGTGGTACCGTTGTTACCGGTCGTGTAGAGCGCGGTATCATCAAAGTAGGTGAAGAAGTTGAAATCGTTGGTATCAAAGAAACCGCGAAAACCACCTGTACTGGCGTTGAAATGTTCCGCAAACTGCTGGACGAAGGCCGTGCTGGTG
@c0mpiler
c0mpiler / SRR1057376.json
Last active April 10, 2019 17:06
search for SRR1057376 on dev cluster
{
"took" : 2795,
"timed_out" : false,
"_shards" : {
"total" : 60,
"successful" : 60,
"skipped" : 0,
"failed" : 0
},
"hits" : {
@c0mpiler
c0mpiler / omxware_spore_genes.json
Created March 12, 2019 22:20
OMXWare Genes with `spore` in the name
{
"took" : 2074,
"hits" : {
"total" : 52979,
},
"aggregations" : {
"names" : {
"buckets" : [
{
"key" : "Spore germination protein YndE",
@c0mpiler
c0mpiler / resorty_numpy_2_cluster_sort_order.py
Created January 10, 2019 21:36
Resort alpha-sorted numpy matrix to cluster-sort order
#!/usr/bin/env python
import numpy as np
import time
from datetime import timedelta
def cluster_sort(genus1, genus2):
start_time = time.monotonic()