Skip to content

Instantly share code, notes, and snippets.

@nreHieW
nreHieW / transformer.py
Created July 9, 2024 13:36
2024 Noam Transformer
"""
The 2024 Transformer (the Noam Transformer):
- RMSNorm
- GQA or some combination
- Sliding window attention
- Swiglu
- RoPE (Rotary Positional Embedding)
LLM Arches:
hidden | MLP mult. | n_layers | rope_theta | GQA Group Size | GLU Act. | ops
@nreHieW
nreHieW / modelling.py
Last active September 29, 2024 15:53
Discounted Cash Flow (DCF) modelling in Python
import pandas as pd
import numpy as np
def calc_cost_of_capital(interest_expense, pre_tax_cost_of_debt, average_maturity, bv_debt, num_shares_outstanding, curr_price, unlevered_beta, tax_rate, risk_free_rate, equity_risk_premium):
market_value_of_debt = interest_expense * (1 - 1 / ((1 + pre_tax_cost_of_debt) ** average_maturity)) / pre_tax_cost_of_debt + bv_debt / ((1 + pre_tax_cost_of_debt) ** average_maturity)
market_value_of_equity = num_shares_outstanding * curr_price
market_value_of_capital = market_value_of_debt + market_value_of_equity
equity_weight = market_value_of_equity / market_value_of_capital
debt_weight = market_value_of_debt / market_value_of_capital