Skip to content

Instantly share code, notes, and snippets.

@hannes
Last active November 15, 2015 18:49
Show Gist options
  • Save hannes/9d26bcf93eff129a036f to your computer and use it in GitHub Desktop.
Save hannes/9d26bcf93eff129a036f to your computer and use it in GitHub Desktop.
MonetDBLite installation and usage example
# Obsolete, See https://www.monetdb.org/blog/monetdblite-r
@hannes
Copy link
Author

hannes commented Sep 28, 2015

See https://www.monetdb.org/blog/monetdblite-r
You can use any directory for the second parameter to dbConnect(), provided its writeable. Then you can also keep the data you loaded around for another time.

@awakmu
Copy link

awakmu commented Sep 29, 2015

Package: MonetDBLite
Version: 1.0.0
Title: In-Process version of MonetDB for R

I still can't believe it yet. You made it in just few months.
Is this really MonetDB server, that embedden in R? Or R embedded in MonetDB server?
So, with MonetDBLite, I can include MonetDBLite library inside my own program?

@hannes
Copy link
Author

hannes commented Sep 29, 2015

@aris it is a MonetDB database, embedded in R. You talk to it via R function calls wrapped a DBI interface. We also had R as a UDF language in MonetDB for a year or so now. But yes, with MonetDBLite, you can embed MonetDB in other programs. We also have a low-level C library, see https://dev.monetdb.org/hg/MonetDB/file/embedded/tools/embedded/embedded.h for the interface.

@iantist
Copy link

iantist commented Nov 12, 2015

Thanks for all the work on the package. I noticed an issue I wanted to get your thoughts on. I have written R code that combs through csv files I want combined into a database table. The csv's are individually read and than appended to a sqlite database using essentially:

tempsheet <- read.csv(ith csv)
dbWriteTable(con, name = "tablename", value = tempsheet, append = TRUE)

As the script runs, the amount of avaliable RAM stays pretty constant. When I change this code to write to a monetdblite database instead of sqlite, the amount of available ram steadily decreases as the script runs. Any ideas on why this might be happening or tips on ways around it?

@hannes
Copy link
Author

hannes commented Nov 15, 2015

@iantist please note that there is a difference between allocated virtual memory and the resident set. MonetDB uses memory-mapped files to store data, this is expected behavior. Are you experiencing any issues due to the raising memory footprint?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment