Scientific Computing Environment for archaeology: Installing R, Rstudio and other things on a lightweight Linux in a VirtualBox virtual machine
Short instructions to setup a Lubuntu Virtual Machine with | |
R & RStudio: | |
1. Download these: | |
http://lubuntu.net/ (Intel x86 desktop cd) | |
https://www.virtualbox.org/wiki/Downloads (Oracle VM VirtualBox) | |
2. Install Oracle VM VirtualBox, open it (if using windows, | |
run as administrator), click 'New' button, at | |
'Name' put Lubuntu, 'Type' choose Linux, 'Version' choose | |
Ubuntu (32-bit), then click 'Next' and change memory size | |
to 2048 MB (or more, but as a rule of thumb not more | |
than a quarter of your total RAM), Click 'Next', select | |
'create virtual drive now', click 'Next', | |
select 'VDI', click 'Next', select 'Fixed size', | |
click 'Next', set disk size to 8GB, click 'Create'. Feel | |
free to make those values larger if you like. If you can't | |
set it to 8GB, set it as large as you can. | |
3. At the main Oracle VM VirtualBox window, double-click on | |
'Lubuntu' the VM you just set up. You will then get a little | |
window that asks you to 'select start-up disk', click on the | |
little folder icon to the right of the drop-down field, then | |
find where you downloaded the lubuntu desktop cd. It should | |
be a file that ends in iso and is about 700 MB. Once you've | |
selected that file, click 'start' on the 'select start-up disk' | |
window. If you get an error, try turning off your antivirus | |
software, if that doesn't help, uninstall VirtualBox and | |
download and install a new copy of VirtualBox | |
from here: https://www.virtualbox.org/wiki/Testbuilds | |
4. You will then be taken to the Lubuntu install menu, | |
choose 'English' by pressing enter, then on the next | |
screen 'Install Lubuntu' (use the arrow keys to select, | |
then press enter). Look in the lower right corner of the | |
window to see what button you need to press to free the | |
mouse (for me it's the right control key). It will take | |
a few minutes to install, keep an eye on it, you will | |
need to click 'continue', then 'download updates while | |
installing', then 'continue' again, then 'erase disk | |
and install Lubuntu', then 'continue', then edit your location | |
then 'continue', then 'continue again (the keyboard | |
layout is usually fine), then type your name and password, | |
keep it easy and simple as this is a very low security | |
use (we're not using this for online banking!) and | |
you'll be typing the password often. Select 'log in | |
automatically' then click 'continue'. Then wait a few | |
minutes and watch the 'copying files' progress bar. | |
Once that's done, it will probably want to restart, | |
click to accept a restart (you may have to hit enter | |
a few times to move the process along), or if you | |
don't get the option, click on the | |
little circular shutdown icon in the lower right | |
corner of the window, and choose 'reboot', and wait | |
for it to restart (hitting enter a few times if | |
it looks like it's stopped doing things). | |
5. Now open a terminal emulator window, which you | |
can do by clicking on the blue button in the lower | |
left corner, then click on 'Accessories' then click | |
on 'LXTerminal'. Or you can press CTRL-ALT-t | |
6. At the terminal prompt, type these, one line | |
at a time, exactly as printed here, and press | |
enter after each line (you will be prompted to | |
type your password, note that you wont see | |
anything on screen when you type in | |
your password, no dots or asterixes - this is | |
normal). Once you've typed your password, press | |
enter. Here are the lines to type in: | |
sudo apt-get update | |
sudo apt-get install build-essential module-assistant -y | |
sudo m-a prepare | |
sudo apt-get install virtualbox-guest-x11 -y | |
Wait for things to scroll past in the console, | |
it will take a few minutes. When you get your | |
prompt back (ie stuff stops scrolling by and | |
the console is ready for a new command), restart | |
your VM by clicking on the little circular | |
shutdown icon in the lower right corner of | |
the window, and choosing 'reboot' or you can | |
type this at the terminal: | |
sudo reboot | |
Once you're going again, look to the main | |
Oracle VM VirtualBox menu bar, click on | |
'Devices', then 'Shared clipboard', and set | |
'Shared Clipboard' to bidirectional. | |
Now you can copy and paste text to and from | |
your VM. | |
Go back to the 'Devices' menu you were just at | |
and click on 'Shared folder settings...' then | |
click on 'Machine Folders', then click on the | |
little button that is a picture of a folder | |
with a plus sign, that will make a little box | |
appear. In this box, click on the little down | |
arrow at the right of the 'Folder Path' field, | |
then click on 'Other...' and choose a folder | |
on your computer that contains files you want | |
to work with in your VM. Check the boxes for | |
'auto-mount' and 'make permanent', then click | |
OK, and click OK again. In your VM, open a terminal | |
and type: | |
sudo usermod -aG vboxsf MYUSERNAME | |
Where MYUSERNAME is the username you set up for | |
yourself when you made the VM. Enter your password | |
when prompted, then reboot the VM again. This will | |
give you access to the folder on your computer | |
from the VM, so you can easily transfer files between | |
the two environments. The shared folder can be | |
found on the VM in a directory called 'media', | |
which you can access by typing /media in the | |
file manager address bar. | |
7. Open a terminal and run these three lines | |
(you can now copy and paste, so do that): | |
wget https://gist.githubusercontent.com/benmarwick/11204658/raw/install_things.sh | |
chmod +x install_things.sh | |
sudo ./install_things.sh | |
then press enter and wait, watching things | |
scroll by. This step downloads a small script | |
from github and runs it. The script installs | |
R, RStudio, and a bunch of other things. | |
This will take 30+ mins, the exact time | |
depends on the speed of your internet | |
connection. Keep an eye on it, you'll | |
need to enter your password here and | |
there, and press enter once or twice. If | |
know what you're doing you can edit the script | |
to add or subtract items. But as it is will be | |
fine for most purposes. | |
8. Now you're ready to compute! If you've | |
never used linux or the terminal before, | |
do check out the video lessons at: | |
https://www.youtube.com/playlist?list=PLA86D04D6E0BFD2E0 | |
and the text at: | |
http://software-carpentry.org/v5/novice/shell/index.html | |
Here are a few tips to get started: | |
sudo firefox # opens a web browser | |
sudo rstudio # opens rstudio | |
sudo gedit # opens a text editor | |
sudo abiword # opens a word processor | |
sudo gnumeric # opens a spreadsheet |
#!/bin/bash | |
# Marwick Lab Computational Environment Setup | |
# this script can be run from the terminal with the next line (minus the #) | |
# bash install_things.sh | |
# you will need to enter your password at a few points, so keep an eye on it while it runs. | |
# in case we need to step through: | |
# set -x | |
# trap read debug | |
## Need this for Debian but not Lubuntu | |
## Make self su: http://stackoverflow.com/a/15067345/1036500 | |
# install sudo to ensure we get access to everything | |
# su -c "aptitude install sudo" | |
# su root | |
## Here we assume your username is ben ... do edit this line if you chose a different username! | |
# sudo adduser ben sudo | |
echo "install a few dependancies for our workflow" | |
sudo apt-get update -y | |
# sudo apt-get upgrade -y | |
sudo apt-get install libgstreamer0.10-0 -y | |
sudo apt-get install libgstreamer-plugins-base0.10-dev -y | |
sudo apt-get install libcurl4-openssl-dev -y | |
sudo apt-get install libssl-dev libssh2-1-dev -y | |
sudo apt-get install libopenblas-base -y | |
sudo apt-get install libxml2-dev -y | |
sudo apt-get install make -y | |
sudo apt-get install gcc -y | |
sudo apt-get install git -y | |
sudo apt-get install pandoc -y | |
sudo apt-get install libjpeg62 -y | |
sudo apt-get install unzip -y | |
sudo apt-get install curl -y | |
sudo apt-get install littler -y | |
sudo apt-get install openjdk-7-* -y | |
sudo apt-get install gedit -y | |
sudo apt-get install jags -y | |
sudo apt-get install imagemagick -y | |
sudo apt-get install docker-engine -y | |
sudo apt-get install gdebi-core | |
# sudo apt-get install mysql-server -y | |
# for Texlive with Lubuntu (making PDFs) | |
# these are BIG, not recommended if you're short on disk space! | |
# sudo add-apt-repository ppa:texlive-backports/ppa | |
# sudo apt-get update | |
# sudo apt-get install texlive-latex-base -y | |
# sudo apt-get install texlive-fonts-recommended -y | |
# sudo apt-get install texlive-latex-extra -y | |
# a few handy FOSS items for drawing, graphics and maps | |
# sudo apt-get install inkscape -y | |
# sudo apt-get install gimp -y | |
# sudo apt-add-repository ppa:ubuntugis/ppa | |
# sudo apt-get update | |
# sudo apt-get install qgis -y | |
# also a few things in case we use python | |
# from http://faculty.washington.edu/rjl/uwhpsc-coursera/vm.html | |
#sudo apt-get install liblzma-dev -y | |
#sudo apt-get install ipython -y | |
#sudo apt-get install ipython-notebook -y | |
#sudo apt-get install python-pandas -y | |
#sudo apt-get install python-numpy -y | |
#sudo apt-get install python-scipy -y | |
#sudo apt-get install python-matplotlib -y | |
#sudo apt-get install python-dev -y | |
#sudo apt-get install python-sphinx -y | |
#sudo apt-get install gfortran -y | |
#sudo apt-get install openmpi-bin -y | |
#sudo apt-get install liblapack-dev -y | |
#sudo apt-get install thunar -y | |
#sudo apt-get install gitk -y | |
#sudo apt-get install docdiff -y | |
#sudo apt-get install python-setuptools -y | |
echo "edit the sources file to prepare to install R" | |
# see http://cran.r-project.org/bin/linux/ubuntu/README | |
sudo sh -c 'echo "deb http://cran.rstudio.com/bin/linux/ubuntu yakkety/" >> /etc/apt/sources.list' # if Lubuntu | |
echo "get keys to install R" | |
# sudo apt-key adv --keyserver keys.gnupg.net --recv-key 381BA480 # if Debian | |
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9 # if Lubuntu | |
echo "install R and some helpers" | |
sudo apt-get update | |
sudo apt-get install r-base -y | |
sudo apt-get install r-base-dev -y | |
sudo apt-get install r-cran-xml -y | |
sudo apt-get install r-cran-rjava -y | |
sudo R CMD javareconf # for rJava | |
echo "install RStudio from the web" | |
# use daily build to get rmarkdown & latest goodies | |
# http://stackoverflow.com/a/15046782/1036500 | |
# check if 32 or 64 bit and install appropriate version... | |
# http://stackoverflow.com/a/106416/1036500 | |
# currently these steps are needed to get RStudio installed https://stackoverflow.com/a/38646838/1036500 | |
MACHINE_TYPE=`uname -m` | |
if [ ${MACHINE_TYPE} == 'x86_64' ]; then | |
# 64-bit stuff here | |
URL=$(wget -q -O - http://www.rstudio.org/download/daily/desktop/ubuntu64 | grep -o -m 1 "https[^\']*" ) | |
FILE=`mktemp`; sudo wget "$URL" -qO $FILE && sudo gdebi $FILE --non-interactive; rm $FILE | |
else | |
# 32-bit stuff here | |
URL=$(wget -q -O - http://www.rstudio.org/download/daily/desktop/ubuntu32 | grep -o -m 1 "https[^\']*" ) | |
FILE=`mktemp`; sudo wget "$URL" -qO $FILE && sudo gdebi $FILE --non-interactive; rm $FILE | |
fi | |
echo "start R and install commonly used packages" | |
# http://stackoverflow.com/q/4090169/1036500 | |
# Make an R script file to use in a moment... | |
LOADSTUFF="options(repos=structure(c(CRAN='http://cran.rstudio.com/'))) | |
update.packages(checkBuilt = TRUE, ask = FALSE) | |
packages <- c('codetools', 'Rcpp', 'devtools', 'knitr', 'ggplot2', 'plyr', 'dplyr', 'XML', 'RCurl', 'readxl', 'tidyr') | |
# just some of my most often used ones | |
install.packages(packages) | |
# and some from github | |
devtools::install_git(c('https://github.com/rstudio/rmarkdown.git', 'https://github.com/hadley/reshape'))" # close the R script | |
# put that R code into an R script file | |
FILENAME1="loadstuff.r" | |
sudo echo "$LOADSTUFF" > /tmp/$FILENAME1 | |
# Make a shell file that contains instructions in bash for running that R script file | |
# from the command line. There may be a simpler way, but nothing I tried worked. | |
NEWBASH='#!/usr/bin/env | |
sudo Rscript /tmp/loadstuff.r' | |
FILENAME2="loadstuff.sh" | |
# put that bash code into a shell script file | |
sudo echo "$NEWBASH" > /tmp/$FILENAME2 | |
# run the bash file to exectute the R code and install the packages | |
sh /tmp/loadstuff.sh | |
# clean up by deleting the temp file | |
rm /tmp/loadstuff.sh | |
# done. | |
echo "all done" | |
echo "type 'sudo rstudio' in the terminal to start RStudio" |
# set cran repo so the user doesn't have to | |
options(repos=structure(c(CRAN='http://cran.rstudio.com/'))) | |
# update installed pacakges | |
update.packages(checkBuilt = TRUE, ask = FALSE) | |
# make a list of additional packages to install | |
packages <- c('codetools', 'Rcpp', 'devtools', 'knitr', 'ggplot2', 'plyr', 'dplyr', 'XML', 'RCurl', 'readxl', 'tidyr') | |
# install that list | |
install.packages(packages) | |
# and some from github | |
devtools::install_git(c('https://github.com/rstudio/rmarkdown.git', 'https://github.com/hadley/reshape'))" | |
message("All done installing additional packages") | |
# end | |
############# More detailed bits and pieces ############## | |
If archeos https://github.com/archeos/ArcheOS gets things right it could replace all of this, just run that in the VM | |
The Berkeley Computational Enviroment http://collaboratool.berkeley.edu/ is similar but a bit heavy | |
## First download a Linux distribution: | |
# Debian internet-installer image file: http://www.debian.org/distrib/netinst | |
# or | |
# Lubuntu, if looking for something smaller and using less memory, use the desktop cd from http://lubuntu.net/ | |
# then download and install Oracle VM VirtualBox: https://www.virtualbox.org/wiki/Downloads | |
## start VirtualBox | |
# Make new VM, follow prompts, set "Type: Linux" and "Version: Debian" or if using Lubuntu, "Version: Ubuntu" then ok | |
# at 'select start-up disk' choose the downloaded iso to make the VM from, then if using Debian, use the Debian graphic installer | |
# many fields can be left blank, try them and see, but DO be sure to select “Debian desktop environment” | |
# There are many useful guides for this (stop following when you get to the desktop, don't run any of the code in these how-tos) | |
# Here's a selection of how-tos (you might also search youtube) | |
# Mac: http://williamjturkel.net/2013/05/31/installing-debian-linux-in-a-vm/ | |
# http://www.brianlinkletter.com/installing-debian-linux-in-a-virtualbox-virtual-machine/ | |
# Windows: https://jtreminio.com/2012/07/setting-up-a-debian-vm-step-by-step/ | |
## install guest additions (GA) to allow for copy and pasting between VM and host | |
# http://askubuntu.com/questions/22743/how-do-i-install-guest-additions-in-a-virtualbox-vm | |
# first open terminal and type (without #) | |
# sudo apt-get update | |
# sudo apt-get install make gcc | |
# then at the terminal, change to the GA directory and type: | |
# sudo ./VBoxLinuxAdditions.run | |
# then | |
# sudo reboot | |
# Once VM has finished installing and Linux is up and running on your screen, | |
# open a terminal to continue, in Debian do: | |
Applications -> Accessories ->Tterminal | |
# Or in Lubuntu do: | |
Button in lower left of screen -> Accessories -> LXTerminal | |
# see shell script below for getting things installed, read through it before running to | |
# know what to expect, then copy-paste the shell script below into the text editor (gedit, leafpad etc.) on your VM, | |
# save locally on your VM, then run the shell script from the terminal by | |
# typing the next line in the terminal (without the #): | |
# bash install_things.sh | |
## Other things we want to do with a VM: | |
# to share the entire Linux VM with others | |
# http://superuser.com/questions/235376/how-to-share-a-virtualbox-image | |
file -> export appliance | |
# to swap files between host OS and guest OS: https://www.virtualbox.org/manual/ch04.html | |
# and don't forget to add self to share folder: https://www.virtualbox.org/ticket/7951 | |
# Debian: getting in and out of su: | |
# in: su - | |
# out: su yourusername | |
# Docker | |
# this seems to be important to set variables and make Docker work on windows | |
# eval "$(docker-machine.exe env docker)" | |
# To do: | |
# add right-click or keyboard shortcut to 'open terminal' cf. http://askubuntu.com/a/207448/119774 http://www.cyberciti.biz/faq/linux-gnome-open-terminal-shell-prompt-here/ | |
# see http://cran.r-project.org/bin/linux/ubuntu/README | |
# sudo sh -c 'echo "deb http://cran.rstudio.com/bin/linux/debian wheezy-cran3/" >> /etc/apt/sources.list' # if Debian | |
sudo sh -c 'echo "deb http://cran.rstudio.com/bin/linux/ubuntu trusty/" >> /etc/apt/sources.list' # if Lubuntu | |
# sudo sh -c 'echo "deb http://cran.rstudio.com/ raring-backports main restricted universe" >> /etc/apt/sources.list' # if Lubuntu | |
# sudo easy_install rpy2 # if using python: now we can sudo ipython notebook & use R within: http://nbviewer.ipython.org/github/ipython/ipython/blob/3607712653c66d63e0d7f13f073bde8c0f209ba8/docs/examples/notebooks/rmagic_extension.ipynb#Rmagic-Functions-Extension | |
From https://github.com/ropensci/docker/blob/master/ropensci/Dockerfile | |
## Add CRAN binaries and update | |
RUN add-apt-repository -y ppa:marutter/c2d4u && apt-get update -qq && apt-get upgrade -qy | |
## From the Build-Depends of the Debian R package: 968 MB! | |
RUN apt-get install -y --no-install-recommends gcc g++ gfortran libblas-dev liblapack-dev tcl8.5-dev tk8.5-dev bison groff-base libncurses5-dev libreadline-dev debhelper texinfo libbz2-dev liblzma-dev libpcre3-dev xdg-utils zlib1g-dev libpng-dev libjpeg-dev libx11-dev libxt-dev x11proto-core-dev libpango1.0-dev libcairo2-dev libtiff5-dev xvfb xauth xfonts-base texlive-base texlive-latex-base texlive-generic-recommended texlive-fonts-recommended texlive-fonts-extra texlive-extra-utils texlive-latex-recommended texlive-latex-extra default-jdk mpack bash-completion | |
## rmarkdown-related dependencies: 146 MB | |
RUN apt-get install -y --no-install-recommends texlive-humanities lmodern texlive-fonts-recommended texlive-latex-extra imagemagick pandoc pandoc-citeproc | |
## R binaries: 149 MB | |
RUN apt-get install -y r-cran-dplyr r-cran-devtools r-cran-rcpp r-cran-xml r-cran-rjava r-cran-rmysql r-cran-rsqlite libxslt1-dev r-cran-rsqlite.extfuns r-cran-rmysql r-cran-data.table r-cran-testthat r-cran-knitr r-cran-ggplot2 r-cran-mgcv r-cran-mass r-cran-tidyr r-cran-shiny r-cran-monetdb.r libxslt1-dev libpq-dev | |
## Not avaiable on marutter/c2d4u | |
# r-cran-rstudioapi r-cran-microbenchmark r-cran-rpostgresql r-cran-roxygen2 | |
RUN echo 'options("repos"="http://cran.rstudio.com", encoding="UTF-8")' > /.Rprofile | |
RUN Rscript -e 'install.packages(c("RPostgreSQL", "roxygen2", "microbenchmark", "rstudioapi"))' | |
RUN Rscript -e 'source("http://bioconductor.org/biocLite.R"); biocLite("rhdf5", ask=FALSE); biocLite("BiocInstaller")' | |
RUN Rscript -e 'devtools::install_github(c("rstudio/rmarkdown", "rstudio/packrat", "hadley/bigrquery", "hadley/reshape"))' | |
RUN Rscript -e 'install.packages(c("SSOAP", "XMLSchema", "Sxslt"), repo="http://www.omegahat.org/R", type="source")' | |
RUN Rscript -e 'devtools::install_github("egonw/rrdf", subdir="rrdflibs", build_vignettes=FALSE); devtools::install_github("egonw/rrdf", subdir="rrdf", build_vignettes=FALSE)' | |
### | |
ubuntu mini, https://help.ubuntu.com/community/Installation/MinimalCD | |
http://ask.xmodulo.com/install-minimal-ubuntu-desktop.html | |
## if buggy, get the pre-built VM from | |
http://www.osboxes.org/lubuntu/ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment