Skip to content

Instantly share code, notes, and snippets.

View sroctadian's full-sized avatar

Syah Reza Octadian sroctadian

7 followers · 3 following
View GitHub Profile
@sroctadian
sroctadian / mitre_attk_as_csv.py
Created November 26, 2021 02:54
Get mitre attack as CSV
from bs4 import BeautifulSoup
import requests
r = requests.get('https://attack.mitre.org/versions/v10/')
soup = BeautifulSoup(r.text)
tables = soup.find_all('table')[1]
fout = open('mitre_attack.csv', 'w')
for tr in tables.find_all('td'):
if tr.a == None:
@sroctadian
sroctadian / analysis.R
Created August 23, 2021 09:39
Plumber endpoint
#' Return the prediction
#' @json
#' @post /predicts
function(req){
predict <- list(status=0, message="success");
predict
}
@sroctadian
sroctadian / analysis-restserver.R
Created August 23, 2021 09:29
Create REST for your analytic or model
library(plumber)
app_dir <- Sys.getenv("ROOT_DIR","/home/apps")
setwd(app_dir)
plumbers <- plumb(
paste0(app_dir, "/analysis.R")
)
plumbers$run(host="127.0.0.1", port=19012)
@sroctadian
sroctadian / build2
Created June 4, 2021 07:33
Create distributed jar with version support
#!/bin/bash
#
# Structure
# .
# jars
# dist.txt
# branch
# default
# [your branch]
#
@sroctadian
sroctadian / yarn-application-list.py
Created December 3, 2020 10:50
Yarn application list command with readable start time and json output
#!/bin/python
import os,json,datetime
CMD_LIST = 'yarn application -list'
CMD_STATUS = 'yarn application -status {}'
toDict = lambda x: { x.split(":")[0].strip(): x.split(":")[1].strip() }
@sroctadian
sroctadian / create_date_dimension.py
Last active May 9, 2019 04:16
import pandas as pd
import calendar
import datetime
def create_date_dimension(start='2019-01-01', end='2020-12-31'):
df = pd.DataFrame({"Date": pd.date_range(start, end)})
df["Day"] = df.Date.dt.weekday_name
df["Week"] = df.Date.dt.weekofyear
df["Quarter"] = df.Date.dt.quarter
@sroctadian
sroctadian / solr-metrics-prometheus-exporter.py
Created January 24, 2019 05:28
Export solr statistics to prometheus metrics
import json
import requests
import time
import prometheus_client
from prometheus_client import start_http_server, Histogram, Gauge, CollectorRegistry
#-global variable
global _proxies, _solr_host, _solr_port, _prometheus_gateway_host, prometheus_gateway_port
_proxies = dict(http='socks5://localhost:9000', https='socks5://localhost:9000')
_use_proxy = False
@sroctadian
sroctadian / gist:09a945bf26a39b966e3d835e4a48f8a0
Created March 6, 2018 09:28
Grook Pattern
#-[http]
#-Access.log
%{IPORHOST:clientip} - - \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent}
@sroctadian
sroctadian / csv2dsv.py
Created October 8, 2017 07:44
Convert .csv to .dsv with pipe separator
#- source : https://pymotw.com/2/csv/
import csv
import sys
f = open(sys.argv[1], 'rt')
fout = open(sys.argv[1] + '.dsv', 'w')
def clean(x):
return x.replace('|', '')
@sroctadian
sroctadian / gist:36a4dcec8952784826be913348beb4f8
Last active November 29, 2016 09:27
How can I spin down external hard drive?
You can safely unmount and spin-down an external hard disk from the terminal most easily by using the command-line functionality of udisks,
which does not require the use of sudo if your system is set up correctly. (To list your device names, enter mount in the terminal first.)
When you have found your external drive, use the following commands.
You must first unmount the partition (use sdb1 or whatever mount showed as the location):
udisks --unmount /dev/sdb1
Then to safely remove (i.e. spindown- you will hear it click and spin-down), use only sdb, for example:
udisks --detach /dev/sdb