Skip to content

Instantly share code, notes, and snippets.

View DmitryBe's full-sized avatar

Dmitry B DmitryBe

View GitHub Profile
@younesbelkada
younesbelkada / bnb-serialization.py
Created December 25, 2023 18:10
push bnb 4 bit models on the hub
# pip install -U bitsandbytes
# pip install -U git+https://github.com/huggingface/transformers.git
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "mistralai/Mixtral-8x7B-Instruct-v0.1"
path_to_hub = XXX
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, load_in_4bit=True)
model.push_to_hub(path_to_hub)
@afranzi
afranzi / SparkWithHadoop.md
Last active February 18, 2024 18:45
Spark 2.4.0 with Hadoop 2.8.5

Setup Environmnet variables for Hadoop.

export HADOOP_VERSION=2.8.5
export HADOOP_HOME=${HOME}/hadoop-$HADOOP_VERSION
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export PATH=${HADOOP_HOME}/bin:$PATH

Download Hadoop files.

@iandanforth
iandanforth / continuous_cartpole.py
Last active May 13, 2024 19:19
Continuous Cartpole for OpenAI Gym
"""
Classic cart-pole system implemented by Rich Sutton et al.
Copied from http://incompleteideas.net/sutton/book/code/pole.c
permalink: https://perma.cc/C9ZM-652R
Continuous version by Ian Danforth
"""
import math
import gym
@zparnold
zparnold / one_liner.sh
Last active June 25, 2024 07:12
A simply script to delete all failed pods from Kubernetes
kubectl get pods --all-namespaces | grep Evicted | awk '{print $2 " --namespace=" $1}' | xargs kubectl delete pod
@EvenOldridge
EvenOldridge / ExportKerasToTensorflowServing.py
Last active January 11, 2019 21:42
Export a Keras Model for Tensorflow Serving
import numpy as np
import tensorflow as tf
import keras as k
from keras.applications.resnet50 import ResNet50
from keras import backend as K
from keras.layers.core import Flatten, Dense, Dropout, Lambda
from keras.models import Model
from keras.preprocessing import image
@jubel-han
jubel-han / pipeline_venv_workarounds.groovy
Created April 13, 2017 08:14
Jenkins pipeline python virtualenv workarounds
node {
stage 'Checkout and Build'
createVirtualEnv 'env'
executeIn 'env', 'pip install -r requirements.txt'
executeIn 'env', './manage.py test'
executeIn 'env', './manage.py integration-test'
virtualEnv('true')
runCmd('pip install -r requirements.txt')
@ruzickap
ruzickap / aws_create_site.yml
Created February 16, 2017 12:35
Ansible playbook which creates instances and tag volumes
---
- name: Create Instance in AWS
hosts: localhost
connection: local
gather_facts: false
vars:
aws_access_key: "xxxxxx"
aws_secret_key: "xxxxxx"
security_token: "xxxxxx"
@alexislucena
alexislucena / install-scala-sbt-and-java-on-ubuntu.md
Created December 5, 2016 12:35
Ubuntu: Install Scala, SBT and Java on Ubuntu 16.04

Install Scala 2.11.8

$ sudo apt-get remove scala-library scala
$ sudo wget www.scala-lang.org/files/archive/scala-2.11.8.deb
$ sudo dpkg -i scala-2.11.8.deb

Check Scala version

$ scala -version
@fancellu
fancellu / MergeSort.scala
Created November 25, 2015 12:12
MergeSort in Scala with recursive merge
object MergeSort {
// recursive merge of 2 sorted lists
def merge(left: List[Int], right: List[Int]): List[Int] =
(left, right) match {
case(left, Nil) => left
case(Nil, right) => right
case(leftHead :: leftTail, rightHead :: rightTail) =>
if (leftHead < rightHead) leftHead::merge(leftTail, right)
else rightHead :: merge(left, rightTail)
@heatxsink
heatxsink / glog-example.go
Last active March 25, 2023 06:02
An example of how to use golang/glog.
/*
glog-example
------------
background
---
You probably want to read the source code comments at the top of the glog.go file in
the golang/glog repository on github.com. Located here: https://github.com/golang/glog/blob/master/glog.go
setup