Skip to content

Instantly share code, notes, and snippets.

@amoeba
Created September 29, 2023 05:28
Show Gist options
  • Save amoeba/38591e99bd8682b60779021ac57f146b to your computer and use it in GitHub Desktop.
Save amoeba/38591e99bd8682b60779021ac57f146b to your computer and use it in GitHub Desktop.
library(arrow)
NROWS <- 500
NCOLS <- 32000
mydf <- as.data.frame(matrix(rnorm(NROWS*NCOLS), nrow=NROWS))
tbl <- as_arrow_table(mydf)
out_path <- tempfile()
write_feather(tbl, out_path)
open_dataset(out_path, format = "arrow") |>
select(1) |>
collect()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment