Skip to content

Instantly share code, notes, and snippets.

View maynewong's full-sized avatar

Mayne Wong maynewong

View GitHub Profile
@ivanfioravanti
ivanfioravanti / gist:bcacc48ef68b02e9b7a4034161824287
Created January 25, 2024 00:01
Fine tuning dataset generation wiht Ollama python library
import json
import os
import ollama
def query_ollama(prompt, model='openhermes:7b-mistral-v2.5-q6_K', context=''):
response = ollama.generate(
model=model,
prompt=context + prompt)
return response['response'].strip()
@andreaseriksson
andreaseriksson / convert_to_verified_routes.ex
Last active March 31, 2024 12:29
This is a mix task for converting old Phoenix routes to new verified routes
defmodule Mix.Tasks.ConvertToVerifiedRoutes do
@shortdoc "Fix routes"
use Mix.Task
@regex ~r/(Routes\.)(.*)_(path|url)\(.*?\)/
@web_module MyAppWeb
def run(_) do
Path.wildcard("lib/**/*.*ex")
@alex9311
alex9311 / LDAspark.md
Last active April 15, 2020 23:13
How to implement LDA in Spark and get the topic distributions of new documents

How to implement LDA in Spark and get the topic distributions of new documents

import org.apache.spark.rdd._
import org.apache.spark.mllib.clustering.{LDA, DistributedLDAModel, LocalLDAModel}
import org.apache.spark.mllib.linalg.{Vector, Vectors}
import scala.collection.mutable

val stopWordsInput = sc.textFile("stopwords.csv")
val stopWords = stopWordsInput.collect()
@staltz
staltz / introrx.md
Last active May 24, 2024 07:56
The introduction to Reactive Programming you've been missing
@bomberstudios
bomberstudios / sketch-plugins.md
Last active February 26, 2024 07:02
A list of Sketch plugins hosted at GitHub, in no particular order.
@waleking
waleking / SparkGibbsLDA.scala
Last active January 31, 2020 11:15
We implement gibbs sampling for LDA by Spark. This version performs much better than alpha version, and now can handle 3196204 words, 100 topics, 1000 sample iterations on server in 161.7 minutes. To solve the long time consuming in collect() process in alpha version, we utilize the cache() method as line 261 and line 262. We also solve a pile o…
package topic
import spark.broadcast._
import spark.SparkContext
import spark.SparkContext._
import spark.RDD
import spark.storage.StorageLevel
import scala.util.Random
import scala.math.{ sqrt, log, pow, abs, exp, min, max }
import scala.collection.mutable.HashMap