Skip to content

Instantly share code, notes, and snippets.

View astariul's full-sized avatar
🧠
100% powered by 3 braincells

Astariul astariul

🧠
100% powered by 3 braincells
View GitHub Profile
@aditya-malte
aditya-malte / smallberta_pretraining.ipynb
Created February 22, 2020 13:41
smallBERTa_Pretraining.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@andyljones
andyljones / mpshang.py
Last active October 28, 2022 09:28
Demonstration of the MPS hang bug
"""This script should be run on a machine with at least 2 GPUs and an MPS server running. You can launch an MPS daemon with
```
nvidia-cuda-mps-control -d
```
The script first uses `test_cuda` to verify a CUDA context can be created on each GPU. It then spawns two workers; a
'good' worker and a 'bad' worker. The workers collaborate through Pytorch's DataDistributedParallel module to calculate
the gradient for a trivial computation. The 'good' worker carries out both the forward and backward pass, while the
bad worker carries out the forward pass and then exits. This seems to lock up the MPS server, and any subsequent
attempts to create CUDA contexts fail by hanging eternally.
@tvst
tvst / SessionState.py
Last active April 14, 2024 20:24
DO NOT USE!!! Try st.session_state instead.
"""Hack to add per-session state to Streamlit.
Usage
-----
>>> import SessionState
>>>
>>> session_state = SessionState.get(user_name='', favorite_color='black')
>>> session_state.user_name
''
@treuille
treuille / confirm_button_hack.py
Last active August 25, 2023 14:38
Hack to implement a confirm_button in Streamlit v0.35
"""
THIS HAS BEEN DEPRECATED!
We propose you use st.form() instead:
https://blog.streamlit.io/introducing-submit-button-and-forms/
"""
import streamlit as st
import collections
@thomwolf
thomwolf / top-k-top-p.py
Last active May 14, 2024 00:20
Sample the next token from a probability distribution using top-k and/or nucleus (top-p) sampling
def top_k_top_p_filtering(logits, top_k=0, top_p=0.0, filter_value=-float('Inf')):
""" Filter a distribution of logits using top-k and/or nucleus (top-p) filtering
Args:
logits: logits distribution shape (vocabulary size)
top_k >0: keep only top k tokens with highest probability (top-k filtering).
top_p >0.0: keep the top tokens with cumulative probability >= top_p (nucleus filtering).
Nucleus filtering is described in Holtzman et al. (http://arxiv.org/abs/1904.09751)
"""
assert logits.dim() == 1 # batch size 1 for now - could be updated for more but the code would be less clear
top_k = min(top_k, logits.size(-1)) # Safety check
@yasufumy
yasufumy / argmax.py
Created December 17, 2016 08:29
argmax function by pure python
def argmax1(array):
return array.index(max(array))
def argmax2(array):
return max(range(len(array)), key=lambda x: array[x])
@iamtekeste
iamtekeste / Download Google Drive files with WGET
Created July 8, 2015 11:00
Download Google Drive files with WGET
Download Google Drive files with WGET
Example Google Drive download link:
https://docs.google.com/open?id=[ID]
To download the file with WGET you need to use this link:
https://googledrive.com/host/[ID]
Example WGET command:
@SpotlightKid
SpotlightKid / getdocstrings.py
Created September 1, 2014 09:22
Parse Python source code and get or print docstrings.
# -*- coding: utf-8 -*-
"""Parse Python source code and get or print docstrings."""
__all__ = ('get_docstrings', 'print_docstrings')
import ast
from itertools import groupby
from os.path import basename, splitext