Skip to content

Instantly share code, notes, and snippets.

@worace
worace / with_stringified_values.sql
Created Apr 18, 2021
stringify non-string values in jsonb
View with_stringified_values.sql
-- migrate:up
CREATE OR REPLACE FUNCTION with_stringified_values(input jsonb)
RETURNS jsonb
LANGUAGE plpgsql AS
$func$
DECLARE
_key text;
_value text;
_vals text[] := '{}'::text[];
_output jsonb := '{}'::jsonb;
@worace
worace / apache-spark.rb
Created Feb 7, 2020 — forked from eddies/apache-spark.rb
Apache Spark 2.2.0 Homebrew
View apache-spark.rb
class ApacheSpark < Formula
desc "Engine for large-scale data processing"
homepage "https://spark.apache.org/"
url "https://archive.apache.org/dist/spark/spark-2.2.0/spark-2.2.0-bin-hadoop2.7.tgz"
version "2.2.0"
sha256 "97fd2cc58e08975d9c4e4ffa8d7f8012c0ac2792bcd9945ce2a561cf937aebcc"
head "https://github.com/apache/spark.git"
bottle :unneeded
@worace
worace / fixed_ish.geojson
Last active Jan 26, 2020
PostGIS tutorial red hook neighborhood example
View fixed_ish.geojson
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View sotm_geoq_lightning_talk.md

Geoq - (yet another) GIS command line tool

  • Problem - ad-hoc manipulation + inspection of piles of textual GIS data
  • Replacing copy-paste workflows
  • Most work is already in the terminal, so unix tools are a plus
    • integrate with other tools
    • streaming enables processing large datasets
    • fast startup + executable binaries are nice
    • Easy, cross-platform install (rust)
  • README - https://github.com/worace/geoq
View usa.geojson
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
View usa.geojson
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@worace
worace / test.geojson
Last active Jul 13, 2018
GeoJSON Example
View test.geojson
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@worace
worace / command_line.md
Created Dec 22, 2017
Command Line Tools and Unix I/O
View command_line.md

Intro: Command Line Tools

  • Types of programs: Side-effect-y (mkdir, touch) vs Function-y (cat, grep, wc) -- Produce output
  • Basic Unix "contract": Take in text stream and output text stream
  • Somewhat janky but works fairly well

Examples

  • head - show me first lines of a file
@worace
worace / accumulator.log
Last active Oct 16, 2017
Accumulators Log Output
View accumulator.log
Compiling accumulators.core
log4j:WARN No appenders could be found for logger (accumulators.core).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
17/10/16 14:28:30 INFO SparkContext: Running Spark version 2.1.0
17/10/16 14:28:30 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/10/16 14:28:30 INFO SecurityManager: Changing view acls to: horace
17/10/16 14:28:30 INFO SecurityManager: Changing modify acls to: horace
17/10/16 14:28:30 INFO SecurityManager: Changing view acls groups to:
@worace
worace / factual_geo.geojson
Last active Oct 6, 2017
Franklin Mountain State Park
View factual_geo.geojson
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.