Skip to content

Instantly share code, notes, and snippets.

View ramnathv's full-sized avatar

Ramnath Vaidyanathan ramnathv

View GitHub Profile
ramnathv /
Created January 1, 2024 00:07 — forked from mlabonne/
Easy Llama 2 fine-tuning script (📝 Article:
# Based on younesbelkada/
# Install the following libraries:
# pip install accelerate==0.21.0 peft==0.4.0 bitsandbytes==0.40.2 transformers==4.31.0 trl==0.4.7 scipy
from dataclasses import dataclass, field
from typing import Optional
import torch
from datasets import load_dataset
from transformers import (
This file has been truncated, but you can view the full file.
{"doc_id":"nXJBccSwtB8_0","video_id":"nXJBccSwtB8","content":"You said these are dangerous times. The world order is shifting before our eyes. We also both know that with hyper disruptive technologies like AI on the horizon, a good outcome is not guaranteed. Why do you think big tech will become the third superpower and what are the dangers and opportunities if it does? Big tech is essentially sovereign over the digital world. The fact that former President Trump was de-platformed from Facebook and from Twitter when he was president, you know, most powerful political figure on the planet. And he's just taken off of those networks and as a consequence, hundreds of millions of people that would be regularly engaging with him in real time suddenly can't see it. That wasn't a decision that was made by a government. It wasn't a decision made by a judge or by a regulatory authority or even by a multinational organization like, you know, the UN. It was made by individuals that own tech companies. The same thing is t
ramnathv /
Created November 29, 2022 18:45 — forked from pydevops/
Google Cloud Platform example to add IAM role restricting user to specific storage buckets with conditions
#!/usr/bin/env bash
export PROJECT_ID=$(gcloud config get-value project)
export PROJECT_USER=$(gcloud config get-value core/account) # set current user
export PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format="value(projectNumber)")
export IDNS=${PROJECT_ID} # workload identity domain
export GCP_REGION="us-central1"
export GCP_ZONE="us-central1-a"
ramnathv /
Created October 27, 2022 11:55 — forked from JustinShenk/
Google Cloud Platform (GCP) instance idle shutdown
# Add to instance metadata with `gcloud compute instances add-metadata \
# instance-name --metadata-from-file` and reboot
# NOTE: requires `bc`, eg, sudo apt-get install bc
# Modified from
while true
# This example demonstrates running furrr code distributed on 2 AWS instances ("nodes").
# The instances have already been created.
# Two t2.micro AWS instances
# Created from
public_ip <- c("", "")
ramnathv / nyc_tlc.Rmd
Created May 21, 2022 18:04 — forked from lambiase/nyc_tlc.Rmd
New York City Taxi & Limousine Commission (TLC) Trip Data Analysis Using Sparklyr and Google BigQuery
title: New York City Taxi & Limousine Commission (TLC) Trip Data Analysis Using Sparklyr
and Google BigQuery
author: "Mirai Solutions"
date: 8\textsuperscript{th} January 2018
theme: flatly
# gcp_json_keyfile: gcp_keyfile.json

App Install Plan


ramnathv /
Created September 8, 2020 04:25 — forked from koreyou/
Implementation of OKapi BM25 with sklearn's TfidfVectorizer
""" Implementation of OKapi BM25 with sklearn's TfidfVectorizer
Distributed as CC-0 (
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from scipy import sparse
class BM25(object):
ramnathv / group_mean.R
Last active July 2, 2019 19:17
#' Group Mean
#' @examples
#' iris %>%
#' group_mean(Species, Petal.Length)
group_mean <- function(tbl, group_var, summary_var){
tbl %>%
group_by({{ group_var }}) %>%
{{ summary_var }} := mean({{ summary_var }})