Skip to content

Instantly share code, notes, and snippets.

View koen-dejonghe's full-sized avatar

Koen Dejonghe koen-dejonghe

View GitHub Profile
@younesbelkada
younesbelkada / finetune_llama_v2.py
Last active May 6, 2024 23:58
Fine tune Llama v2 models on Guanaco Dataset
# coding=utf-8
# Copyright 2023 The HuggingFace Inc. team. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
// MARK: - Some differentiable array manipulation functions used in the algorithms.
extension Array where Element: Differentiable {
@differentiable(vjp: _vjpSwappedAt)
func swappedAt(_ i: Int, _ j: Int) -> Array {
var tmp = self
tmp.swapAt(i, j)
return tmp
}
@snakers4
snakers4 / calculate_knn.py
Created August 26, 2018 12:00
Use faiss to calculate a KNN graph on data
import gc
import tqdm
import faiss
import bcolz
import os,sys
import numpy as np
from tqdm import tqdm
# open the stored bcolz array
# note that these vectors have to be 280 dimensional
@davideicardi
davideicardi / PassThroughFlow.scala
Last active June 6, 2022 08:32
Akka stream generic pass through flow. For latest implementation see https://developer.lightbend.com/docs/alpakka/current/patterns.html#passthrough
/*
https://scalafiddle.io/sf/sniohcZ/3
(old version: https://scalafiddle.io/sf/sniohcZ/1)
Use PassThroughFlow when you have a message that should be used in a
flow that trasform it but you want to maintain the original message for
another following flow.
For example if you consume messages from Kafka (CommittableMessage).
You process the message (transform, save it inside a database, ...) and then you need again the original message
to commit the offset.
@koen-dejonghe
koen-dejonghe / DinosaurIslandCharRnn.scala
Last active February 20, 2018 08:07
Dinosaurus Island -- Character level language model
package scorch.examples
import botkop.numsca.Tensor
import botkop.{numsca => ns}
import scorch.autograd._
import scorch.nn.rnn.RecurrentModule
import scorch.nn.Optimizer
import scala.annotation.tailrec
import scala.io.Source
@lattner
lattner / TaskConcurrencyManifesto.md
Last active May 11, 2024 08:56
Swift Concurrency Manifesto
case class SomeEvent(value: Long)
val events = Source
.tick(0 seconds, 250 millis, "")
.zipWithIndex
.map { case (_, l) =>
SomeEvent(l)
}
val group = Flow[SomeEvent].groupedWithin(100, 500 millis) // +/- 2 events per group
@adamw
adamw / windowing.scala
Created August 5, 2016 13:30
Windowing data in Akka
package com.softwaremill.akka
import java.time._
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.stream.scaladsl.Source
import scala.collection.mutable
import scala.concurrent.Await
@kylemcdonald
kylemcdonald / showarray.py
Created January 3, 2016 08:56
Minimal code for rendering a numpy array as an image in a Jupyter notebook in memory. Borrowed from the Deep Dream notebook.
import PIL.Image
from cStringIO import StringIO
import IPython.display
import numpy as np
def showarray(a, fmt='png'):
a = np.uint8(a)
f = StringIO()
PIL.Image.fromarray(a).save(f, fmt)
IPython.display.display(IPython.display.Image(data=f.getvalue()))