Skip to content

Instantly share code, notes, and snippets.

View sayef's full-sized avatar

Md Saiful I. Sayef sayef

View GitHub Profile
@sayef
sayef / pytorch-positional-encoding.py
Created May 4, 2023 10:11
PyTorch Positional Encoding
import math
from torch import nn, Tensor
class PositionalEncoding(nn.Module):
def __init__(self, embedding_dim: int, dropout: float = 0.1, max_len: int = 5000):
super().__init__()
self.dropout = nn.Dropout(p=dropout)
position = torch.arange(max_len).unsqueeze(1)
div_term = torch.exp(
  • If the bucket is not full, add the record.
  • Otherwise, split the bucket.
    • Allocate a new leaf and move half the bucket elements to the new bucket.
    • Insert the new leaf's smallest key and address into the parent.
    • If the parent is full, split it too.
      • Add the middle key to the parent node.
    • Repeat until a parent is found that need not split.
  • If the root splits, create a new root that has one key and two pointers. (That is, the value that gets pushed to the new root gets removed from the original node)
@sayef
sayef / large-file-hash.py
Created June 29, 2022 20:01 — forked from aunyks/large-file-hash.py
Hash a large file in Python
import hashlib as hash
# Specify how many bytes of the file you want to open at a time
BLOCKSIZE = 65536
sha = hash.sha256()
with open('kali.iso', 'rb') as kali_file:
file_buffer = kali_file.read(BLOCKSIZE)
while len(file_buffer) > 0:
sha.update(file_buffer)
@sayef
sayef / gitsshproxy.md
Created March 30, 2022 20:27
How to use git ssh command using ssh-proxy
  1. create a ssh-proxy ssh -D 9999 -qCN user@server.net

  2. make git connect through the ssh-proxy

  • [current script only] export GIT_SSH_COMMAND='ssh -o ProxyCommand="connect -S 127.0.0.1:9999 %h %p"'
  • OR [git global setting] git config --global core.sshCommand 'ssh -o ProxyCommand="connect -S 127.0.0.1:9999 %h %p"'
  • OR [one-time only use] git clone -c=core.sshCommand 'ssh -o ProxyCommand="connect -S 127.0.0.1:9999 %h %p"' git@github.com:user/repo.git
{
"cardinal number": [
"A Russian diver has found the bodies of three of the [E] 118 [/E] sailors who were killed when the nuclear submarine Kursk sank in the Barents Sea .",
"Navy officials do not expect that all [E] 118 [/E] bodies can be recovered .",
"In the latest daily CNN USA Today Gallup popular opinion survey , George W Bush , the Governor of Texas , the Republican nominee , holds a 49 % to 42 % advantage over his Democratic opponent Al Gore , the Vice President , but [E] two [/E] other major surveys gave Mr. Bush only a one - point lead .",
"In the latest daily CNN USA Today Gallup popular opinion survey , George W Bush , the Governor of Texas , the Republican nominee , holds a 49 % to 42 % advantage over his Democratic opponent Al Gore , the Vice President , but two other major surveys gave Mr. Bush only a [E] one [/E] - point lead .",
"The paper concludes that there is [E] one [/E] candidate who will do that and that is George W Bush .",
"The bat
@sayef
sayef / python-utility-functions.md
Created September 11, 2021 20:23
Python Utility Functions
  1. Convert list to bytes and get back list again:
import numpy as np
from io import BytesIO

def list2bytes(v):
    np_array = np.array(v)
    np_bytes = BytesIO()
 np.save(np_bytes, np_array, allow_pickle=True)
@sayef
sayef / download_latest_release_file.sh
Last active September 11, 2021 14:06
Download latest release from github repo
LATEST=$(curl -s https://github.com/$1/releases | grep -ioE $1"/archive/refs/tags/LMDB_.*tar.gz" | head -n 1)
wget https://github.com/$LATEST
@sayef
sayef / compile.sh
Last active August 28, 2021 13:59
Clean Python Code with autoflake, autopep8 and isort
# add this file in .vscode folder as compile.sh
# edit ~/.local/bin/ if it's different in your machine
FILE=$(wslpath $1)
EXT=$(rev <<< "$FILE" | cut -d"." -f1 | rev)
if [ "$EXT" == "py" ]; then
echo "Running autopep8 and autoflake on $FILE $EXT"
~/.local/bin/autopep8 -i -a -a $FILE
~/.local/bin/autoflake -i --remove-all-unused-imports --remove-unused-variables $FILE
@sayef
sayef / download_svg.js
Created May 7, 2021 17:08
Download SVG Node From HTML
# specify your svg element and run this in chrome console
var svg = document.getElementsByTagName("svg")[0];
var serializedSVG = new XMLSerializer().serializeToString(svg);
var base64Data = window.btoa(serializedSVG);
var dataURL = "data:image/svg+xml;base64," + base64Data
var dl = document.createElement("a");
document.body.appendChild(dl); // This line makes it work in Firefox.
dl.setAttribute("href", dataURL);
dl.setAttribute("download", "test.svg");
dl.click();
@sayef
sayef / ctc_score.py
Created May 6, 2021 11:40 — forked from githubharald/ctc_score.py
Compute confidence score for CTC-decoded text using TensorFlow
"""
Compute score for decoded text in a CTC-trained neural network using TensorFlow:
1. decode text with best path decoding (or some other decoder)
2. feed decoded text into loss function
3. loss is negative logarithm of probability
Example data: two time-steps, 2 labels (0, 1) and the blank label (2).
Decoding results in [0] (i.e. string containing one entry for label 0).
The probability is the sum over all paths yielding [0], these are: [0, 0], [0, 2], [2, 0]
with probability