Skip to content

Instantly share code, notes, and snippets.

@Thomascountz
Created September 14, 2021 02:53
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save Thomascountz/4ccb5adbf76694672ab37db67a62404a to your computer and use it in GitHub Desktop.
Save Thomascountz/4ccb5adbf76694672ab37db67a62404a to your computer and use it in GitHub Desktop.
Columnar Datastore?
#!/usr/bin/env ruby
## USAGE EXAMPLE
## $ bin/coloco '{ "key": "value" }'
##
## $ find ./datastore -type f | xargs tail -n +1
## ==> ./datastore/key <==
## value
##
## ==> ./datastore/index <==
## 1
Dir["./lib/*.rb"].each { |file| require file }
Coloco.write(ARGV[0])
# lib/coloco.rb
require 'json'
require 'fileutils'
class Coloco
DATASTORE_DIR = "datastore"
INDEX_FILE = "index"
def self.write(raw_event)
event = JSON.parse(raw_event)
existing_columns = Dir.children(DATASTORE_DIR)
index_filepath = File.join(DATASTORE_DIR, INDEX_FILE)
head = File.open(index_filepath, 'r') { |index| index.gets.to_i }
new_columns = (event.keys - existing_columns)
new_columns.each do |column|
filepath = File.join(DATASTORE_DIR, column)
File.open(filepath, 'a') do |file|
head.times { file.puts nil }
end
end
Dir.each_child(DATASTORE_DIR) do |column|
filepath = File.join(DATASTORE_DIR, column)
File.open(filepath, 'a') do |file|
file.puts event[column]
end
end
File.open(index_filepath, 'w') do |index|
index.puts head.next
end
end
end
#!/bin/bash
mkdir -p ./datastore
rm ./datastore/*
echo -ne 0 >> ./datastore/index
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment