Skip to content

Instantly share code, notes, and snippets.

View ibnesayeed's full-sized avatar

Sawood Alam ibnesayeed

93 followers · 1 following
View GitHub Profile
@ibnesayeed
ibnesayeed / randomlines.py
Created May 26, 2021 18:45
Stream random lines from a file
#!/usr/bin/env python3
import sys
from random import randrange
class RandomLines():
def __init__(self, fname):
self._fname = fname
@ibnesayeed
ibnesayeed / ipwb-cov-report.md
Created August 26, 2020 16:32
IPWB Test Coverage Report 
/tmp/ipwb$ coverage run -m pytest
=================================== test session starts ===================================
platform linux -- Python 3.8.2, pytest-6.0.1, py-1.9.0, pluggy-0.13.1
rootdir: /tmp/ipwb, configfile: setup.cfg
plugins: flake8-1.0.6, cov-2.10.1
collected 121 items                                                                                

tests/test_backends.py .....                                                        [  4%]
tests/test_compile_target_uri.py ..                                                 [  5%]
@ibnesayeed
ibnesayeed / README.md
Last active May 1, 2020 22:47
Mount your remote machine's home directory locally over SSHFS

SSHFS Remote Home Mounter

Download the script, make it executable, and run:

$ chmod a+x remotehomemount.sh
$ ./remotehomemount.sh [<REMOTEID> [<MNTDIR> [<REMOTEHOST>]]]
@ibnesayeed
ibnesayeed / README.md
Last active March 26, 2020 04:36
PyWB Memento Compliance Audit Data

PyWB Memento Compliance Audit Data

Test data file pywbtest.tar.gz has the following directory structure:

$ tree pywbtest
├── collections
│   └── example
│       ├── archive
│ │ ├── example-20200323133704.warc.gz
@ibnesayeed
ibnesayeed / docker-build.md
Last active November 6, 2019 22:00 
$ ls
Dockerfile  linkextractor.py

$ cat linkextractor.py
#!/usr/bin/env python

import sys
@ibnesayeed
ibnesayeed / twitter-timemap-lang-downloader.sh
Last active April 17, 2019 21:39
A Shell script to download TimeMaps of a Twitter handle in various language variants using MemGator
#!/usr/bin/env bash
# A script to download TimeMaps of a Twitter handle in various language variants.
#
# ./twitter-timemap-lang-downloader.sh <TwitterHandle> [<OutputDir>]
#
# Author: Sawood Alam <@ibnesayeed>
if [ $# -eq 0 ]
then
@ibnesayeed
ibnesayeed / slides.txt
Created November 28, 2018 20:59
CS531 Guest Lecture Slides
https://www.cs.odu.edu/~salam/presentations/HTTPMethodSupport.pdf
https://www.cs.odu.edu/~salam/presentations/WebARChiveFileFormat.pdf
https://www.cs.odu.edu/~salam/httpmailbox/presentation.html
@ibnesayeed
ibnesayeed / Dockerfile
Created August 15, 2018 15:05
Piping in Dockerfile CMD for IPWB with customizable WARC file using an environment variable
FROM oduwsdl/ipwb
ENV WARC=ipwb/samples/warcs/5mementos.warc
CMD ipfs daemon & while ! curl -s localhost:5001 > /dev/null; do sleep 1; done && ipwb index $WARC | ipwb replay
@ibnesayeed
ibnesayeed / Dockerfile
Created August 15, 2018 04:20
Piping in Dockerfile CMD for IPWB
FROM oduwsdl/ipwb
CMD ipfs daemon & while ! curl -s localhost:5001 > /dev/null; do sleep 1; done && ipwb index ipwb/samples/warcs/5mementos.warc | ipwb replay
@ibnesayeed
ibnesayeed / twitter-barackobama-capture-languages.tsv
Created March 22, 2018 23:00
Twitter BarackObama Capture Language Distribution
We can't make this file beautiful and searchable because it's too large.
DateTime Archive Status Language URIM
20070312000128 IA 200 en http://web.archive.org/web/20070312000128/http://twitter.com:80/BarackObama
20070312000213 IA 200 en http://web.archive.org/web/20070312000213/http://twitter.com:80/barackobama
20070320110428 IA 200 en http://web.archive.org/web/20070320110428/http://twitter.com:80/BarackObama
20070429014820 IA 200 en http://web.archive.org/web/20070429014820/http://twitter.com:80/barackobama
20070505120209 IA 200 en http://web.archive.org/web/20070505120209/http://twitter.com:80/BarackObama
20070513015443 IA 200 en http://web.archive.org/web/20070513015443/http://twitter.com:80/BarackObama
20070513141310 IA 200 en http://web.archive.org/web/20070513141310/http://twitter.com:80/BarackObama
20070514045148 IA 200 en http://web.archive.org/web/20070514045148/http://twitter.com:80/BarackObama
20070524090236 IA 200 en http://web.archive.org/web/20070524090236/http://twitter.com:80/BarackObama