Created
September 14, 2021 02:53
-
-
Save Thomascountz/4ccb5adbf76694672ab37db67a62404a to your computer and use it in GitHub Desktop.
Columnar Datastore?
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env ruby | |
## USAGE EXAMPLE | |
## $ bin/coloco '{ "key": "value" }' | |
## | |
## $ find ./datastore -type f | xargs tail -n +1 | |
## ==> ./datastore/key <== | |
## value | |
## | |
## ==> ./datastore/index <== | |
## 1 | |
Dir["./lib/*.rb"].each { |file| require file } | |
Coloco.write(ARGV[0]) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# lib/coloco.rb | |
require 'json' | |
require 'fileutils' | |
class Coloco | |
DATASTORE_DIR = "datastore" | |
INDEX_FILE = "index" | |
def self.write(raw_event) | |
event = JSON.parse(raw_event) | |
existing_columns = Dir.children(DATASTORE_DIR) | |
index_filepath = File.join(DATASTORE_DIR, INDEX_FILE) | |
head = File.open(index_filepath, 'r') { |index| index.gets.to_i } | |
new_columns = (event.keys - existing_columns) | |
new_columns.each do |column| | |
filepath = File.join(DATASTORE_DIR, column) | |
File.open(filepath, 'a') do |file| | |
head.times { file.puts nil } | |
end | |
end | |
Dir.each_child(DATASTORE_DIR) do |column| | |
filepath = File.join(DATASTORE_DIR, column) | |
File.open(filepath, 'a') do |file| | |
file.puts event[column] | |
end | |
end | |
File.open(index_filepath, 'w') do |index| | |
index.puts head.next | |
end | |
end | |
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
mkdir -p ./datastore | |
rm ./datastore/* | |
echo -ne 0 >> ./datastore/index |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment