Skip to content

Instantly share code, notes, and snippets.

From main:

Benchmark Time CPU Iterations items_per_second
AllocateDeallocate/size:4096/real_time/threads:1 40.6 ns 40.5 ns 17256246 24.6592M/s
AllocateDeallocate/size:4096/real_time/threads:2 37.1 ns 74.3 ns 18816032 26.9313M/s
AllocateDeallocate/size:4096/real_time/threads:4 68.5 ns 274 ns 10096888 14.5911M/s
AllocateDeallocate/size:4096/real_time/threads:8 85.2 ns 682 ns 8969680 11.733M/s
AllocateDeallocate/size:4096/real_time/threads:16 64.1 ns 1022 ns 10904176 15.5995M/s
AllocateDeallocate/size:4096/real_time/threads:32 57.1 ns 1033 ns 13711680 17.5085M/s
require 'net/http'
require "arrow"
url = URI('http://localhost:9292')
Net::HTTP.start("localhost", 9292) do |http|
req = Net::HTTP::Get.new(url)
http.request(req) do |res|
res.read_body do |chunk|
Gio::RubyInputStream.open(chunk)
@amoeba
amoeba / streaming.rb
Created March 13, 2024 22:01
WIP version of streaming record batches with red-arrow
require "arrow"
nrows = 0
batches = []
Net::HTTP.start(host, port) do |http|
req = Net::HTTP::Get.new(url)
# Note: This works but isn't actually streaming anything, it just looks like it
http.request(req) do |res|
2024-03-06T22:41:18.5774870Z Current runner version: '2.314.1'
2024-03-06T22:41:18.5801470Z ##[group]Operating System
2024-03-06T22:41:18.5802070Z macOS
2024-03-06T22:41:18.5802370Z 12.7.3
2024-03-06T22:41:18.5802680Z 21H1015
2024-03-06T22:41:18.5803000Z ##[endgroup]
2024-03-06T22:41:18.5803350Z ##[group]Runner Image
2024-03-06T22:41:18.5803770Z Image: macos-12
2024-03-06T22:41:18.5804140Z Version: 20240218.1
2024-03-06T22:41:18.5805290Z Included Software: https://github.com/actions/runner-images/blob/macOS-12/20240218.1/images/macos/macos-12-Readme.md
#' Get a list of git tags
#'
#' @param pattern
#'
#' @return
#'
#' @examples
get_tags <- function(pattern=NULL) {
all_tags <- system2("git", c("tag", "--sort=taggerdate"), stdout = TRUE)
#include "arrow/array/array_base.h"
#include "arrow/record_batch.h"
#include "arrow/result.h"
#include <iostream>
#include <memory>
#include <stddef.h>
#include <arrow/api.h>
#include <arrow/compute/api.h>
#' new_contributors.R
#'
#' Produce a list of names of new contributors between two git refs. The method
#' this uses is to first get the list of unique contrbutors referancable from
#' the first ref, then the second ref, and then compute the set difference and
#' return that.
#'
#' Usage
#'
#' Launch an R session from the directory containing the git repo you want to
@amoeba
amoeba / filter_mmaped.cc
Last active October 26, 2023 23:35
Arrow C++ example filtering a random set of rows from a memory-mapped Arrow IPC file using Acero
#include <iostream>
#include <memory>
#include <random>
#include <string>
#include <vector>
#include "arrow/acero/exec_plan.h"
#include "arrow/acero/options.h"
#include "arrow/acero/util.h"
#include "arrow/array/data.h"
library(arrow)
NROWS <- 500
NCOLS <- 32000
mydf <- as.data.frame(matrix(rnorm(NROWS*NCOLS), nrow=NROWS))
tbl <- as_arrow_table(mydf)
out_path <- tempfile()
write_feather(tbl, out_path)
@amoeba
amoeba / output.log
Created September 14, 2023 19:07
R CMD INSTALL output
❯ ARROW_R_DEV=0 R CMD INSTALL --preclean --no-multiarch .
Welcome to R :)
This session's PID is 98315
* installing to library ‘/Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library’
* installing *source* package ‘arrow’ ...
** using staged installation
*** Trying Arrow C++ in ARROW_HOME: /Users/bryce/builds/arrow-arm64
Welcome to R :)
This session's PID is 99418
**** Not using: C++ library version (13.0.0) does not match R package (13.0.0.9000)