This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def round_ste(z): | |
"""Round with straight through gradients.""" | |
zhat = jnp.round(z) | |
return z + jax.lax.stop_gradient(zhat - z) | |
class FSQ: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import math | |
import torch | |
def hadamard(n: int, dtype=torch.int8): | |
"""This function is a port of the one in scipy.linalg""" | |
if n < 1: | |
lg2 = 0 | |
else: | |
lg2 = int(math.log(n, 2)) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import math | |
import torch | |
import torch.optim | |
class CAME(torch.optim.Optimizer): | |
"""Implements CAME algorithm. | |
This implementation is based on: | |
`CAME: Confidence-guided Adaptive Memory Efficient Optimization` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
"""Block-State Transformer Layer.""" | |
# Block Transformers are non-recurrent and parallelizable. | |
block_transformer = jax.vmap(BRecT.nonrecurrent_cell) | |
def BST(u): | |
"""Block-State Transformer Layer.""" | |
global MF # True if Multi-Filter, False otherwise (SH/MH) | |
# split inputs into windows (l/w, w, d) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
""" | |
Simplified Implementation of the Linear Recurrent Unit | |
------------------------------------------------------ | |
We present here a simplified JAX implementation of the Linear Recurrent Unit (LRU). | |
The state of the LRU is driven by the input $(u_k)_{k=1}^L$ of sequence length $L$ | |
according to the following formula (and efficiently parallelized using an associative scan): | |
$x_{k} = \Lambda x_{k-1} +\exp(\gamma^{\log})\odot (B u_{k})$, | |
and the output is computed at each timestamp $k$ as follows: $y_k = C x_k + D u_k$. | |
In our code, $B,C$ follow Glorot initialization, with $B$ scaled additionally by a factor 2 | |
to account for halving the state variance by taking the real part of the output projection. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import torch | |
import numpy as np | |
# numpy | |
a = np.random.rand(10, 20) | |
tmp0 = np.split(a, indices_or_sections=5, axis=0) # split into 5 sections | |
for t in tmp0: | |
print(t.shape) | |
# (2, 20) | |
# (2, 20) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[Script Info] | |
; Script generated by Aegisub 9010-makepkg-6f546951b | |
; http://www.aegisub.org/ | |
ScriptType: v4.00+ | |
WrapStyle: 0 | |
ScaledBorderAndShadow: yes | |
YCbCr Matrix: TV.709 | |
PlayResX: 1280 | |
PlayResY: 720 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<!DOCTYPE html> | |
<html> | |
<head> | |
<title>SNScrape Viewer</title> | |
<script src="https://unpkg.com/mustache@latest"></script> | |
<script src="https://cdn.tailwindcss.com"></script> | |
<script> | |
function checkVisible(elm, threshold, mode) { | |
threshold = threshold || 0; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gen-username() { | |
vowel() { | |
head /dev/urandom | tr -dc 'aeiueoy' | cut -c1 | |
} | |
consonant() { | |
head /dev/urandom | tr -dc 'qwrtpsdfghjklzxcvbnm' | cut -c1 | |
} | |
digit() { | |
head /dev/urandom | tr -dc '[:digit:]' | cut -c1 | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"title": "Made in Abyss - 01", | |
"duration": 10, | |
"live": false, | |
"sources": [ | |
{ | |
"url": "https://cloud1.userscloud.com/d/giluki2xtn2fvxijjto4veyddzoaqowmz6atpvgsxyzkfbknhkvlwc2656llywni2ft46kwd/video.mp4", | |
"contentType": "video/mp4", | |
"quality": 1080, | |
"bitrate": 9000 |
NewerOlder