Skip to content

Instantly share code, notes, and snippets.

View bitsnaps's full-sized avatar
🌍
Working @ CorpoSense

Ibrahim H. bitsnaps

🌍
Working @ CorpoSense
View GitHub Profile
@bitsnaps
bitsnaps / finetune_llama_v2.py
Created April 16, 2024 18:01 — forked from younesbelkada/finetune_llama_v2.py
Fine tune Llama v2 models on Guanaco Dataset
# coding=utf-8
# Copyright 2023 The HuggingFace Inc. team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
@bitsnaps
bitsnaps / dataset_clustering.py
Last active April 9, 2024 12:11
Clustering using AgglomerativeClustering and silhouette scoring
# Create a file "dataset.csv" with these values (don't forget to remove the '#' before each line):
# ID,Height,time_of_day,resolution
# 272,1.567925,1.375000,0.594089
# 562,1.807508,1.458333,0.594089
# 585,2.693542,0.416667,0.594089
# 610,1.036305,1.458333,0.594089
# 633,1.117111,0.416667,0.594089
# 658,1.542407,1.458333,0.594089
# 681,1.930844,0.416667,0.594089
# 802,1.505548,1.458333,0.594089
@bitsnaps
bitsnaps / text_similarity
Created April 20, 2023 15:45
A small script to benchmark different models for text similarity using SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity
try:
import spacy
except:
!pip install spacy
import spacy
try:
from sentence_transformers import SentenceTransformer, util
@bitsnaps
bitsnaps / OAI_CONFIG_LIST
Created January 7, 2024 18:28 — forked from langecrew/OAI_CONFIG_LIST
Taking the Autogen Teachable Agent one step further with some customization
[
{
"model": "gpt-4",
"api_key": "PASTE_YOUR_API_KEY_HERE"
},
{
"model": "gpt-4-1106-preview",
"api_key": "PASTE_YOUR_API_KEY_HERE"
},
{
@bitsnaps
bitsnaps / gptq_lora.py
Created December 19, 2023 18:35 — forked from datasciencemonkey/gptq_lora.py
train a gptq model using peft/lora
# %%
# this is run from /notebooks on paperspace
from huggingface_hub import login
from dotenv import load_dotenv
load_dotenv("/notebooks/.env")
import os
os.environ["TOKENIZERS_PARALLELISM"]="false"
login(token=os.getenv("HUGGINGFACE_TOKEN"))
1. Install oh-my-zsh
sh -c "$(wget https://raw.githubusercontent.com/robbyrussell/oh-my-zsh/master/tools/install.sh -O -)"
2. Clone necessary plugins.
git clone git://github.com/zsh-users/zsh-autosuggestions $ZSH_CUSTOM/plugins/zsh-autosuggestions
git clone https://github.com/zsh-users/zsh-history-substring-search ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-history-substring-search
git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting
3. Add plugins to ~/.zshrc as
plugins = ( [plugins...] zsh-autosuggestions zsh-history-substring-search zsh-syntax-highlighting)
@bitsnaps
bitsnaps / dirsize.js
Last active November 20, 2023 13:19
Get directory size (used in stackblitz as a replacement of `du -sh` command)
import { join } from 'path';
import { readdir, stat } from 'fs/promises';
function humanFileSize(bytes, si = false, dp = 1) {
const thresh = si ? 1000 : 1024;
if (Math.abs(bytes) < thresh) {
return bytes + ' B';
}
@bitsnaps
bitsnaps / psv_decryptor.py
Last active September 27, 2023 22:10
Decrypt PSV file for mac
# This script is inspired from: https://github.com/kigaita/PsvDecryptCore
# It allows you to decrypt psv video files for online developer training website
# The original code was written with .NET for Windows users, this one with Python tested for Mac users.
# *** Disclaimer ****
# Please only use it for your convenience so that you can watch the courses on your devices offline or for educational purposes.
# Piracy is strictly prohibited. Decrypted videos should not be uploaded to open servers, torrents, or other methods of mass distribution. Any consequences resulting from misuse of this tool are to be taken by the user.
# packages you may need to install:
# pip install python-magic
@bitsnaps
bitsnaps / commands.sh
Last active September 21, 2023 18:30
Replit & NixOS cheat sheet
# NixOS is a Linux distribution built on top of the Nix package manager, known for its declarative and reproducible approach to system configuration.
# read more: https://nixos.org/
# Replit uses NixOS under the hood in order with Nix which is a tool for managing packages and system configurations.
#https://docs.replit.com/programming-ide/nix-on-replit
# Update nix packages
nix-channel --update
# list all nix channels
@bitsnaps
bitsnaps / dataframe_viewer.py
Created June 25, 2023 18:09
A simple Qt5 python app to display a CSV or Excel in TableWidget
# pip install openpyxl, pandas, PyQt5... (if you don't use conda)
import sys
import pandas as pd
from PyQt5.QtCore import QSize, Qt, pyqtSlot, pyqtSignal
from PyQt5.QtWidgets import (QApplication, QMainWindow, QPushButton, QTableWidget, QTableWidgetItem,
QFileDialog, QVBoxLayout, QWidget, QDialog, QLabel, QLineEdit, QComboBox, QCheckBox, QHBoxLayout)
# Custom TableWidget class
class TableWidget(QTableWidget):
def __init__(self, df, parent=None):