Skip to content

Instantly share code, notes, and snippets.

View napsternxg's full-sized avatar
🎯
Focusing

Shubhanshu Mishra napsternxg

🎯
Focusing
246 followers · 14 following
View GitHub Profile
@napsternxg
napsternxg / awesome-movies-compare.markdown
Created April 16, 2023 16:39
Awesome Movies Compare
@napsternxg
napsternxg / spacy_transformer.py
Last active April 15, 2023 06:21
Space sklearn Transformer - Use spacy embeddings in Sklearn model pipelines
"""Spacy Embedding Transformer for Sklearn pipeline
Install spacy and floret
```bash
pip install spacy floret scikit-learn
```
First download the vectors from:
```bash
@napsternxg
napsternxg / PyTorchBiggraph-Pytorch-pyarrow.diff
Created April 11, 2023 02:41
Improve edgelist processing speed of PyTorchBiggraph-Pytorch using pyarrow parquet reader.
diff --git a/torchbiggraph/converters/importers.py b/torchbiggraph/converters/importers.py
index fa84bc6..765e9fa 100644
--- a/torchbiggraph/converters/importers.py
+++ b/torchbiggraph/converters/importers.py
@@ -28,6 +28,7 @@ from torchbiggraph.graph_storages import (
RELATION_TYPE_STORAGES,
)
from torchbiggraph.types import UNPARTITIONED
+from tqdm import tqdm
@napsternxg
napsternxg / convergence_to_eigenvector.py
Created April 4, 2023 16:06
Repeated matrix multiplication makes the column vectors converge to eigen vectors of the matrix
# ! pip install celluloid
import numpy as np
import matplotlib.pyplot as plt
from celluloid import Camera
def plot_mat(A, evals=None, evecs=None, fig=None, ax=None):
if ax is None:
fig, ax = plt.subplots(1, 2, figsize=(8, 4))
A = A / np.linalg.norm(A, axis=0, keepdims=True)
ax[0].imshow(A)
@napsternxg
napsternxg / parse_tweetnerd_files.py
Created February 3, 2023 04:16
Parse TweetNERD files
from zipfile import ZipFile
from pathlib import Path
import pandas as pd
import numpy as np
import json
JOB_FILES = list(Path(".").glob("**/job_*.json.zip"))
JOB_ID_TO_OUTPUT_PART = {
1873084: 12,
@napsternxg
napsternxg / ShaderToy-Render.ipynb
Last active August 15, 2022 05:45
RenderShaderToy Example in Google Colab
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@napsternxg
napsternxg / Summarization-Corpora.md
Last active August 10, 2022 03:34
Summarization corpora taken from: https://docs.google.com/spreadsheets/d/1b1-NpM1jDK7KVHd_CwrxhpNZ1zAE8m-7M0pZ0gfZTMQ/edit#gid=0

| Large-scale dataset? | Year of release | Name | Reference | URL to ref | URL to data | Access | Price | License

@napsternxg
napsternxg / LatexDiffOverleaf.md
Last active May 7, 2022 17:22
Doing diff for latex projects on Overleaf

Steps for doing latexdiff

  • Download 2 versions from overleaf
  • Compile them using the TexStudio. I should create a bbl file.
  • Create a new tex file where you replace the bibliography section with the contents of the bbl file.
  • Do the same for the other version of the file.
  • Check that the new tex file for both version compiles to the same contents of the original tex files.
  • Open https://3142.nl/latex-diff/ and paste the contents of the new tex files from both versions in the text boxes.
  • Click the box named "Generate LateX document showing differences"
  • It will take some time and then create the diff file.
@napsternxg
napsternxg / grid_dist.py
Created January 19, 2022 20:55
Grid Distance Function
"""
# Grid Dist
what's a loss function that incorporates spatial distance?
e.g. i want...
loss(y..., y...) == 0
0 < loss(yt, y1) < loss(yt, y2)
loss(yt, y2) == loss(yt, y2.T)
Tweet: https://twitter.com/TheShubhanshu/status/1482132766607265796
"""
@napsternxg
napsternxg / Wordle_Solution.md
Last active January 14, 2022 00:51
Wordle One Shot Solution (Hack)

Wordle One Shot Solution (Hack)

NOTE: This is a hack to get the wordle solution

NOTE: Use it only when you are frustrated and just want to see the solution. It will ruin the fun of the game for you.

I am not doing anything fancy. It is just that the solution is always available on the page and I am reading it and firing the keypresses to get answer.

Be reminded that you should not copy JS from strangers on your pages, it can be used to steal your passwords or other sensitive information. You can read the full code used here and more details at: https://gist.github.com/napsternxg/a80021d6aec9d362db8c736bc2f9a888 Learn about internet security from A PROJECT OF THE ELECTRONIC FRONTIER FOUNDATION: SURVEILLANCE SELF-DEFENSE: TIPS, TOOLS AND HOW-TOS FOR SAFER ONLINE COMMUNICATIONS course from https://ssd.eff.org/en