Skip to content

Instantly share code, notes, and snippets.

@hannes
Last active November 15, 2015 18:49
Show Gist options
  • Save hannes/9d26bcf93eff129a036f to your computer and use it in GitHub Desktop.
Save hannes/9d26bcf93eff129a036f to your computer and use it in GitHub Desktop.
MonetDBLite installation and usage example
# Obsolete, See https://www.monetdb.org/blog/monetdblite-r
@iantist
Copy link

iantist commented Nov 12, 2015

Thanks for all the work on the package. I noticed an issue I wanted to get your thoughts on. I have written R code that combs through csv files I want combined into a database table. The csv's are individually read and than appended to a sqlite database using essentially:

tempsheet <- read.csv(ith csv)
dbWriteTable(con, name = "tablename", value = tempsheet, append = TRUE)

As the script runs, the amount of avaliable RAM stays pretty constant. When I change this code to write to a monetdblite database instead of sqlite, the amount of available ram steadily decreases as the script runs. Any ideas on why this might be happening or tips on ways around it?

@hannes
Copy link
Author

hannes commented Nov 15, 2015

@iantist please note that there is a difference between allocated virtual memory and the resident set. MonetDB uses memory-mapped files to store data, this is expected behavior. Are you experiencing any issues due to the raising memory footprint?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment