Skip to content

Instantly share code, notes, and snippets.

@appliedml42
appliedml42 / mha.py
Last active January 7, 2022 15:39
MultiHeadAttention Implementation using einops and Pytorch
'''
I am reading this amazing series(https://uvadlc-notebooks.readthedocs.io/en/latest). I always struggle with revisiting
my old code that has a lot of tensor manipulation. Experimented with reimplementing their MultiHeadAttention layer using
einops syntax that feels more human readable.
'''
import torch
import torch.nn as nn
import torch.nn.functional as F
import einops
import math
@machuu
machuu / WSL2_VPN_Workaround_Instructions.md
Last active April 18, 2024 23:13
Workaround for WSL2 network broken on VPN

Overview

Internet connection and DNS routing are broken from WSL2 instances, when some VPNs are active.

The root cause seems to be that WSL2 and the VPN use the same IP address block, and the VPN routing clobbers WSL2's network routing.

This problem is tracked in multiple microsoft/WSL issues including, but not limited to:

@barseghyanartur
barseghyanartur / debugging_python_within_docker.md
Last active March 28, 2024 03:43
Debugging Python within Docker

Debugging Python within Docker

Often you need to run debugger within docker. It many cases it looks like this:

import pdb; pdb.set_trace()

Or: