Skip to content

Instantly share code, notes, and snippets.

View ssharkov03's full-sized avatar
🏠
Working from home

Sergey Sharkov ssharkov03

🏠
Working from home
  • Wildberries
  • Moscow
  • 23:21 (UTC +07:00)
View GitHub Profile
@ssharkov03
ssharkov03 / robust_causal_lm_loss.py
Last active May 15, 2025 05:08
Composable loss module that adds Bayesian Invariant Risk Minimization (BIRM) regularization to any causal language model. Plug it into your training loop to encourage domain-invariant representations without touching model code.
"""
The module provides three public classes that implement *Bayesian Invariant Risk Minimization* (BIRM)
training objectives for causal language‑model fine-tuning:
* :class:`EnvironmentEMB` – a lightweight environment embedding that is
randomly perturbed at every forward pass to approximate the posterior over
domain‑specific linear classifiers.
* :class:`BIRMCoefficientScheduler` – a helper that dynamically rescales the
BIRM penalty so that its contribution to the total loss remains in a desired
ratio to the empirical risk (ERM) term.
subreddit_name = 'marvel'
num_comments = 500 # the number of comments you want to get
import requests
import pandas as pd
from datetime import datetime
# we use this function to convert responses to dataframes
def df_from_response(res):
# initialize temp dataframe for batch of data in response
@ssharkov03
ssharkov03 / model_config.py
Created August 18, 2021 19:05
AlexNet model mixed
model_mixed = models.alexnet(pretrained=True)
layers_to_unfreeze = 5
for param in model_mixed.features[:-layers_to_unfreeze].parameters():
param.requires_grad = False
num_features = 9216
model_mixed.classifier = nn.Sequential(
nn.Dropout(p=0.5, inplace=False),
nn.Linear(num_features, 2, bias=True)
@ssharkov03
ssharkov03 / augmentation.py
Created August 18, 2021 18:44
Augmentation for dataset
data_transforms = {
'train': transforms.Compose([
transforms.Resize(600),
transforms.ColorJitter(brightness=.3, hue=.5, contrast=.1, saturation=.1),
transforms.RandomRotation(degrees=30),
transforms.RandomPerspective(distortion_scale=.3, p=.7),
transforms.RandomHorizontalFlip(p=.5),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]),
transforms.RandomErasing(p=.3, scale=(0.01, 0.02))