Skip to content

Instantly share code, notes, and snippets.

View mzpqnxow's full-sized avatar

AG mzpqnxow

View GitHub Profile
@mzpqnxow
mzpqnxow / bulkupsert.py
Created February 5, 2021 19:05 — forked from luke/bulkupsert.py
I needed to upsert (insert or update) bajillions of records into postgresql. After trying various libs including upsert (which was slow as hell) I ended up doing a bit of research and trying 3 different methods. This one won. While I'm manually building the sql string no user data is passed in. Its loaded via the copy from statement as CSV. Call…
import logging
import cStringIO
import csv
DEBUG = False
def data2csv(data):
si = cStringIO.StringIO()
cw = csv.writer(si, delimiter='\t',lineterminator="\n")
for row in data:
## FOR UBUNTU
Dependencies install
1. apt-get install nginx-extras
2. apt-get install lua-zlib
lua file decompress request body
--------------------------------
see https://gist.github.com/iammehrabalam/30f5402bbcdad139c9eafd3a6f47ce6c
@mzpqnxow
mzpqnxow / ubuntu-xenial-armfh-qemu.md
Created September 25, 2020 05:48 — forked from takeshixx/ubuntu-xenial-armfh-qemu.md
Running Ubuntu 16.04.1 armhf on Qemu

Running Ubuntu 16.04.1 armhf on Qemu

This is a writeup about how to install Ubuntu 16.04.1 Xenial Xerus for the 32-bit hard-float ARMv7 (armhf) architecture on a Qemu VM via Ubuntu netboot.

The setup will create a Ubuntu VM with LPAE extensions (generic-lpae) enabled. However, this writeup should also work for non-LPAE (generic) kernels.

The performance of the resulting VM is quite good, and it allows VMs with >1G ram (compared to 256M on versatilepb and 1G on versatile-a9/versatile-a15). It also supports virtio disks whereas versatile-a9/versatile-a15 only support SD cards via the -sd argument.

Get netboot files

# If you're looking into the C10M problem (10 million concurrent connections)
# you might want to play with DPDK (Originally proprietry Intel, now open source)
#
# C10M: http://c10m.robertgraham.com/
# DPDK: http://dpdk.org/
#
# This is a quick summary how to install dpdk on ubuntu
# running inside virtualbox on a mac
# On my Mac:
@mzpqnxow
mzpqnxow / ratelimit.nginxconf
Last active April 2, 2020 22:01 — forked from ipmb/ratelimit.nginxconf
Nginx reverse proxy with rate limiting
#
# This will ratelimit requests to 10/s
# It will allow up to 128 requests to connect, make their request, and then delay the response
# On the client side, this will appear to be "hung", until one of the 10 slots opens up, at
# which point it will proxy your request
#
# If more than 128 requests are queued, any after that will immediately be returned with an
# error
#
#
@mzpqnxow
mzpqnxow / firefox.sh
Created March 22, 2020 21:07 — forked from payloadartist/firefox.sh
Enumerate sub-domains, then open them in Firefox automatically. Useful for taking a quick glance at target's assets, and make notes, while doing recon.
# ------Instructions---------
# Install (and configure) subfinder, assetfinder, and httprobe
# go get -v github.com/projectdiscovery/subfinder/cmd/subfinder && go get -v github.com/tomnomnom/httprobe && go get -v github.com/tomnomnom/assetfinder
# cat firefox.sh >> ~/.bashrc
# source ~/.bashrc
# Usage - subf_ff target.tld
# asset_ff target.tld
subf_ff () {
subfinder -d $1 -silent -t 100 | httprobe -c 50 | sort -u | while read line; do firefox $line; sleep 10; done
@mzpqnxow
mzpqnxow / insert_paginator.py
Created March 19, 2020 13:38 — forked from evan-burke/insert_paginator.py
psycopg2 execute_values wrapper for accurate row counts
# One of the fastest ways to insert bulk data into Postgres (at least, aside from COPY) is using the psycopg2 extras function execute_values.
# However, this doesn't return an accurate row count value - instead, it just returns the row count for the last page inserted.
# This wraps the execute_values function with its own pagination to return an accurate count of rows inserted.
# Performance is approximately equivalent to underlying execute_values function - within 5-10% or so in my brief tests.
import psycopg2
import psycopg2.extras
import math
db_connection_string = "dbname=EDITME host=EDITME"
@mzpqnxow
mzpqnxow / grafana_pdf_via_snapshot.md
Created December 2, 2019 00:35 — forked from svet-b/grafana_pdf_via_snapshot.md
PDF export of Grafana dashboard via snapshot

Get PDF export of Grafana dashboard

Note

As currently described, this approach does not work. The reason for that is that creating a snapshot via the API does not actually populate any data, as described in e.g. https://community.grafana.com/t/snapshot-using-http-api-does-nothing/. An alternative approach which does work described in https://gist.github.com/svet-b/1ad0656cd3ce0e1a633e16eb20f66425.

Prerequisites

Packages:

  • The jq JSON processing command line tool, which is available for most distributions (https://stedolan.github.io/jq/, sudo apt install jq)
  • NodeJS, and the puppeteer package (npm install puppeteer), which is used to run headless Chrome
@mzpqnxow
mzpqnxow / burpscript.py
Created April 19, 2019 17:38 — forked from mwielgoszewski/burpscript.py
This extension registers an IHttpListener configured to execute a custom script editable via the Script tab added to Burp. The script is executed in the context with the following global and local variables (extender, callbacks, helpers, toolFlag, messageIsRequest, messageInfo).
from java.awt import Font
from javax.swing import JScrollPane, JTextPane
from javax.swing.text import SimpleAttributeSet
from burp import IBurpExtender, IExtensionStateListener, IHttpListener, ITab
import base64
import traceback
@mzpqnxow
mzpqnxow / pyscripter_utils.py
Created April 19, 2019 17:10 — forked from lanmaster53/pyscripter-snippets.py
Burp Python Scripter scripts
from burp import IScanIssue
class CustomIssue(IScanIssue):
def __init__(self, BasePair, Confidence='Certain', IssueBackground=None, IssueDetail=None, IssueName='Python Scripter generated issue', RemediationBackground=None, RemediationDetail=None, Severity='High'):
self.HttpMessages=[BasePair] # list of HTTP Messages
self.HttpService=BasePair.getHttpService() # HTTP Service
self.Url=BasePair.getUrl() # Java URL
self.Confidence = Confidence # "Certain", "Firm" or "Tentative"
self.IssueBackground = IssueBackground # String or None
self.IssueDetail = IssueDetail # String or None