Skip to content

Instantly share code, notes, and snippets.

@ScotterC
ScotterC / chunkify.py
Created February 28, 2023 16:25
Chunk up text using lanchain's text splitters
#!/usr/bin/env python3
# Note: if spacy isn't working you may need to download english model python -m spacy download en
import argparse
import json
from langchain.text_splitter import NLTKTextSplitter, CharacterTextSplitter, RecursiveCharacterTextSplitter, SpacyTextSplitter, TokenTextSplitter
def chunkify(text, chunk_size, chunk_overlap, method):
if method == 'nltk':
text_splitter = NLTKTextSplitter.from_tiktoken_encoder(
@ScotterC
ScotterC / blackhole.rb
Created March 23, 2017 17:03
blackhole.rb
#! /usr/bin/ruby
require 'fileutils'
path = "/etc/hosts"
sites = %w(
news.ycombinator.com
reddit.com
www.reddit.com
@ScotterC
ScotterC / gist:5729366
Created June 7, 2013 13:44
Basic mockup of combining Filepicker.io with Paperclip & Delayed Paperclip
View
<%= form_for image do |f| %>
<%= link_to "Pick File", "#", class: 'filepicker' %>
<%= f.hidden_field :remote_url %>
<%= f.hidden_field :filepicker_url %>
<%= f.hidden_field :attachment_file_size %>
<%= f.hidden_field :attachment_file_name %>
<%= f.hidden_field :attachment_content_type %>
<% end %>
@ScotterC
ScotterC / gist:b738721015de6c56573f
Created November 4, 2015 20:58
Basic vagrant error output
~/vagrant-setup (master): vagrant plugin install vagrant-hostsupdater
Installing the 'vagrant-hostsupdater' plugin. This can take a few minutes...
Bundler, the underlying system Vagrant uses to install plugins,
reported an error. The error is shown below. These errors are usually
caused by misconfigured plugin installations or transient network
issues. The error from Bundler is:
An error occurred while installing json (1.8.1), and Bundler cannot continue.
Make sure that `gem install json -v '1.8.1'` succeeds before bundling.
@ScotterC
ScotterC / gist:6703521
Last active February 9, 2016 09:51
Discourse on Rubber

Short Version:

  • git clone Discourse
  • gem install rubber
  • rubber vulcanize discourse
  • Edit rubber.yml
  • cap rubber:create_staging and you've got a fully functioning discourse site

###Long Version:

  • git clone https://github.com/discourse/discourse.git
@ScotterC
ScotterC / gist:7604440
Last active December 29, 2015 02:59
Unicorn initialization script
#!/bin/sh
set -e
# Example init script, this can be used with nginx, too,
# since nginx and unicorn accept the same signals
# Feel free to change any of the following variables for your app:
TIMEOUT=${TIMEOUT-60}
APP_ROOT=/home/x/my_app/current
PID=$APP_ROOT/tmp/pids/unicorn.pid
CMD="/usr/bin/unicorn -D -c $APP_ROOT/config/unicorn.rb"
@ScotterC
ScotterC / gist:7234355
Last active December 26, 2015 23:59
StartupInstitude
class Me
def name
@name
end
def name=(name)
@name = name
end
@ScotterC
ScotterC / gist:5857944
Last active December 18, 2015 22:59
Largest Prime factor of 1e12 number
// Works for 13195 and larger numbers like 901234567 but fails when I add a digit let alone for 600851475143.
// The prime factors of 13195 are 5, 7, 13 and 29
// What is the largest prime factor of the number 600851475143
package main
import (
"fmt"
)
@ScotterC
ScotterC / gist:5482191
Created April 29, 2013 15:05
Hash deep find correction
# A common method you'll find on the internet for a deep_find in a Hash
def deep_find(key)
key?(key) ? self[key] : self.values.inject(nil) {|memo, v| memo ||= v.deep_find(key) if v.respond_to?(:deep_find) }
end
# Let's break this up without the ternarys
def deep_find(key)
if key?(key)
self[key]
else
@ScotterC
ScotterC / gist:5467778
Created April 26, 2013 14:34
Find paperclip meta for images without meta
Image.where(attachment_meta: nil).each do |img|
meta = {}
Image.attachment_definitions[:attachment][:styles].merge({:original => nil}).keys.each do |style|
url = img.attachment.url(style)
uploaded_file = URI.parse(url)
begin
file = Paperclip.io_adapters.for(uploaded_file)