Skip to content

Instantly share code, notes, and snippets.

View syedarehaq's full-sized avatar

Syed Arefinul Haque syedarehaq

View GitHub Profile
@syedarehaq
syedarehaq / mysqldump_to_csv
Created June 11, 2021 20:50
A slightly modified script to convert mysql dump to csv. Utilizing it for the sql dump of wikipedia interwiki links and pagelinks found here: https://dumps.wikimedia.org/enwiki/latest/ . It is slightly modified to accommodate some special unicode characters from its original source found here: https://github.com/jamesmishra/mysqldump-to-csv
#!/usr/bin/env python
## Source: https://github.com/jamesmishra/mysqldump-to-csv
import fileinput
import csv
import sys
# This prevents prematurely closed pipes from raising
# an exception in Python
from signal import signal, SIGPIPE, SIG_DFL
signal(SIGPIPE, SIG_DFL)
@syedarehaq
syedarehaq / elastic_scroll.py
Created October 22, 2020 04:16 — forked from zobayer1/elastic_scroll.py
Simple generator function to download Elasticsearch index data using scroll query
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import requests
class ElasticScroll(object):
"""Manages scroll contexts for elasticsearch scroll queries.
Args:
host (str): Elasticsearch host url. Example: ``http://localhost:9200``.
index (str): Elasticsearch index name. Example: ``my_index``.
@syedarehaq
syedarehaq / keybase.md
Created October 2, 2020 12:36
proving Github identity for keybase

Keybase proof

I hereby claim:

  • I am syedarehaq on github.
  • I am syedarehaq (https://keybase.io/syedarehaq) on keybase.
  • I have a public key ASDBn1wJZyLT4H2_N7hJ_flZa2Eprc6sEYbt1Lyb84Sk7Qo

To claim this, I am signing this object:

@syedarehaq
syedarehaq / README.md
Created April 14, 2020 04:31 — forked from jrladd/README.md
Marvel Network: A Tricked-Out D3 Implementation

This force-directed graph takes advantage of the new features of D3 version 4 to display and manipulate a network of Marvel Comics characters. Click "open" to use the full suite of tools.

Features

  • Scroll to zoom.
  • Use the slider to change the edge-weight threshold.
  • Click on nodes to see ego networks (click again to see all nodes).
  • Use the dropdown to show three different centrality measures, calculated using NetworkX in Python and imported through the marvel.json file.
@syedarehaq
syedarehaq / fileseek.py
Created February 20, 2020 21:52
Seeking a file at a specific byte position or location, usually useful to debug a BiguQuery export error
import argparse
parser = argparse.ArgumentParser()
parser = argparse.ArgumentParser(description="go to the line of a specific position")
parser.add_argument("input_fname", help = "The input csv file that we will be reading")
parser.add_argument("pos", help="position where we will be seeking the line", type=float)
args = parser.parse_args()
fname = args.input_fname
position = args.pos
try:
with open(fname) as file: # don't you go correcting me on naming it file. we don't call file directly anyway!