Last active
May 14, 2018 19:32
-
-
Save benmarwick/11204658 to your computer and use it in GitHub Desktop.
Scientific Computing Environment for archaeology: Installing R, Rstudio and other things on a lightweight Linux in a VirtualBox virtual machine
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Short instructions to setup a Lubuntu Virtual Machine with | |
R & RStudio: | |
1. Download these: | |
http://lubuntu.net/ (Intel x86 desktop cd) | |
https://www.virtualbox.org/wiki/Downloads (Oracle VM VirtualBox) | |
2. Install Oracle VM VirtualBox, open it (if using windows, | |
run as administrator), click 'New' button, at | |
'Name' put Lubuntu, 'Type' choose Linux, 'Version' choose | |
Ubuntu (32-bit), then click 'Next' and change memory size | |
to 2048 MB (or more, but as a rule of thumb not more | |
than a quarter of your total RAM), Click 'Next', select | |
'create virtual drive now', click 'Next', | |
select 'VDI', click 'Next', select 'Fixed size', | |
click 'Next', set disk size to 8GB, click 'Create'. Feel | |
free to make those values larger if you like. If you can't | |
set it to 8GB, set it as large as you can. | |
3. At the main Oracle VM VirtualBox window, double-click on | |
'Lubuntu' the VM you just set up. You will then get a little | |
window that asks you to 'select start-up disk', click on the | |
little folder icon to the right of the drop-down field, then | |
find where you downloaded the lubuntu desktop cd. It should | |
be a file that ends in iso and is about 700 MB. Once you've | |
selected that file, click 'start' on the 'select start-up disk' | |
window. If you get an error, try turning off your antivirus | |
software, if that doesn't help, uninstall VirtualBox and | |
download and install a new copy of VirtualBox | |
from here: https://www.virtualbox.org/wiki/Testbuilds | |
4. You will then be taken to the Lubuntu install menu, | |
choose 'English' by pressing enter, then on the next | |
screen 'Install Lubuntu' (use the arrow keys to select, | |
then press enter). Look in the lower right corner of the | |
window to see what button you need to press to free the | |
mouse (for me it's the right control key). It will take | |
a few minutes to install, keep an eye on it, you will | |
need to click 'continue', then 'download updates while | |
installing', then 'continue' again, then 'erase disk | |
and install Lubuntu', then 'continue', then edit your location | |
then 'continue', then 'continue again (the keyboard | |
layout is usually fine), then type your name and password, | |
keep it easy and simple as this is a very low security | |
use (we're not using this for online banking!) and | |
you'll be typing the password often. Select 'log in | |
automatically' then click 'continue'. Then wait a few | |
minutes and watch the 'copying files' progress bar. | |
Once that's done, it will probably want to restart, | |
click to accept a restart (you may have to hit enter | |
a few times to move the process along), or if you | |
don't get the option, click on the | |
little circular shutdown icon in the lower right | |
corner of the window, and choose 'reboot', and wait | |
for it to restart (hitting enter a few times if | |
it looks like it's stopped doing things). | |
5. Now open a terminal emulator window, which you | |
can do by clicking on the blue button in the lower | |
left corner, then click on 'Accessories' then click | |
on 'LXTerminal'. Or you can press CTRL-ALT-t | |
6. At the terminal prompt, type these, one line | |
at a time, exactly as printed here, and press | |
enter after each line (you will be prompted to | |
type your password, note that you wont see | |
anything on screen when you type in | |
your password, no dots or asterixes - this is | |
normal). Once you've typed your password, press | |
enter. Here are the lines to type in: | |
sudo apt-get update | |
sudo apt-get install build-essential module-assistant -y | |
sudo m-a prepare | |
sudo apt-get install virtualbox-guest-x11 -y | |
Wait for things to scroll past in the console, | |
it will take a few minutes. When you get your | |
prompt back (ie stuff stops scrolling by and | |
the console is ready for a new command), restart | |
your VM by clicking on the little circular | |
shutdown icon in the lower right corner of | |
the window, and choosing 'reboot' or you can | |
type this at the terminal: | |
sudo reboot | |
Once you're going again, look to the main | |
Oracle VM VirtualBox menu bar, click on | |
'Devices', then 'Shared clipboard', and set | |
'Shared Clipboard' to bidirectional. | |
Now you can copy and paste text to and from | |
your VM. | |
Go back to the 'Devices' menu you were just at | |
and click on 'Shared folder settings...' then | |
click on 'Machine Folders', then click on the | |
little button that is a picture of a folder | |
with a plus sign, that will make a little box | |
appear. In this box, click on the little down | |
arrow at the right of the 'Folder Path' field, | |
then click on 'Other...' and choose a folder | |
on your computer that contains files you want | |
to work with in your VM. Check the boxes for | |
'auto-mount' and 'make permanent', then click | |
OK, and click OK again. In your VM, open a terminal | |
and type: | |
sudo usermod -aG vboxsf MYUSERNAME | |
Where MYUSERNAME is the username you set up for | |
yourself when you made the VM. Enter your password | |
when prompted, then reboot the VM again. This will | |
give you access to the folder on your computer | |
from the VM, so you can easily transfer files between | |
the two environments. The shared folder can be | |
found on the VM in a directory called 'media', | |
which you can access by typing /media in the | |
file manager address bar. | |
7. Open a terminal and run these three lines | |
(you can now copy and paste, so do that): | |
wget https://gist.githubusercontent.com/benmarwick/11204658/raw/install_things.sh | |
chmod +x install_things.sh | |
sudo ./install_things.sh | |
then press enter and wait, watching things | |
scroll by. This step downloads a small script | |
from github and runs it. The script installs | |
R, RStudio, and a bunch of other things. | |
This will take 30+ mins, the exact time | |
depends on the speed of your internet | |
connection. Keep an eye on it, you'll | |
need to enter your password here and | |
there, and press enter once or twice. If | |
know what you're doing you can edit the script | |
to add or subtract items. But as it is will be | |
fine for most purposes. | |
8. Now you're ready to compute! If you've | |
never used linux or the terminal before, | |
do check out the video lessons at: | |
https://www.youtube.com/playlist?list=PLA86D04D6E0BFD2E0 | |
and the text at: | |
http://software-carpentry.org/v5/novice/shell/index.html | |
Here are a few tips to get started: | |
sudo firefox # opens a web browser | |
sudo rstudio # opens rstudio | |
sudo gedit # opens a text editor | |
sudo abiword # opens a word processor | |
sudo gnumeric # opens a spreadsheet |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/bash | |
# Marwick Lab Computational Environment Setup | |
# this script can be run from the terminal with the next line (minus the #) | |
# bash install_things.sh | |
# you will need to enter your password at a few points, so keep an eye on it while it runs. | |
# in case we need to step through: | |
# set -x | |
# trap read debug | |
## Need this for Debian but not Lubuntu | |
## Make self su: http://stackoverflow.com/a/15067345/1036500 | |
# install sudo to ensure we get access to everything | |
# su -c "aptitude install sudo" | |
# su root | |
## Here we assume your username is ben ... do edit this line if you chose a different username! | |
# sudo adduser ben sudo | |
echo "install a few dependancies for our workflow" | |
sudo apt-get update -y | |
# sudo apt-get upgrade -y | |
sudo apt-get install libgstreamer0.10-0 -y | |
sudo apt-get install libgstreamer-plugins-base0.10-dev -y | |
sudo apt-get install libcurl4-openssl-dev -y | |
sudo apt-get install libssl-dev libssh2-1-dev -y | |
sudo apt-get install libopenblas-base -y | |
sudo apt-get install libxml2-dev -y | |
sudo apt-get install make -y | |
sudo apt-get install gcc -y | |
sudo apt-get install git -y | |
sudo apt-get install pandoc -y | |
sudo apt-get install libjpeg62 -y | |
sudo apt-get install unzip -y | |
sudo apt-get install curl -y | |
sudo apt-get install littler -y | |
sudo apt-get install openjdk-7-* -y | |
sudo apt-get install gedit -y | |
sudo apt-get install jags -y | |
sudo apt-get install imagemagick -y | |
sudo apt-get install docker-engine -y | |
sudo apt-get install gdebi-core | |
# sudo apt-get install mysql-server -y | |
# for Texlive with Lubuntu (making PDFs) | |
# these are BIG, not recommended if you're short on disk space! | |
# sudo add-apt-repository ppa:texlive-backports/ppa | |
# sudo apt-get update | |
# sudo apt-get install texlive-latex-base -y | |
# sudo apt-get install texlive-fonts-recommended -y | |
# sudo apt-get install texlive-latex-extra -y | |
# a few handy FOSS items for drawing, graphics and maps | |
# sudo apt-get install inkscape -y | |
# sudo apt-get install gimp -y | |
# sudo apt-add-repository ppa:ubuntugis/ppa | |
# sudo apt-get update | |
# sudo apt-get install qgis -y | |
# also a few things in case we use python | |
# from http://faculty.washington.edu/rjl/uwhpsc-coursera/vm.html | |
#sudo apt-get install liblzma-dev -y | |
#sudo apt-get install ipython -y | |
#sudo apt-get install ipython-notebook -y | |
#sudo apt-get install python-pandas -y | |
#sudo apt-get install python-numpy -y | |
#sudo apt-get install python-scipy -y | |
#sudo apt-get install python-matplotlib -y | |
#sudo apt-get install python-dev -y | |
#sudo apt-get install python-sphinx -y | |
#sudo apt-get install gfortran -y | |
#sudo apt-get install openmpi-bin -y | |
#sudo apt-get install liblapack-dev -y | |
#sudo apt-get install thunar -y | |
#sudo apt-get install gitk -y | |
#sudo apt-get install docdiff -y | |
#sudo apt-get install python-setuptools -y | |
echo "edit the sources file to prepare to install R" | |
# see http://cran.r-project.org/bin/linux/ubuntu/README | |
sudo sh -c 'echo "deb http://cran.rstudio.com/bin/linux/ubuntu yakkety/" >> /etc/apt/sources.list' # if Lubuntu | |
echo "get keys to install R" | |
# sudo apt-key adv --keyserver keys.gnupg.net --recv-key 381BA480 # if Debian | |
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys E084DAB9 # if Lubuntu | |
echo "install R and some helpers" | |
sudo apt-get update | |
sudo apt-get install r-base -y | |
sudo apt-get install r-base-dev -y | |
sudo apt-get install r-cran-xml -y | |
sudo apt-get install r-cran-rjava -y | |
sudo R CMD javareconf # for rJava | |
echo "install RStudio from the web" | |
# use daily build to get rmarkdown & latest goodies | |
# http://stackoverflow.com/a/15046782/1036500 | |
# check if 32 or 64 bit and install appropriate version... | |
# http://stackoverflow.com/a/106416/1036500 | |
# currently these steps are needed to get RStudio installed https://stackoverflow.com/a/38646838/1036500 | |
MACHINE_TYPE=`uname -m` | |
if [ ${MACHINE_TYPE} == 'x86_64' ]; then | |
# 64-bit stuff here | |
URL=$(wget -q -O - http://www.rstudio.org/download/daily/desktop/ubuntu64 | grep -o -m 1 "https[^\']*" ) | |
FILE=`mktemp`; sudo wget "$URL" -qO $FILE && sudo gdebi $FILE --non-interactive; rm $FILE | |
else | |
# 32-bit stuff here | |
URL=$(wget -q -O - http://www.rstudio.org/download/daily/desktop/ubuntu32 | grep -o -m 1 "https[^\']*" ) | |
FILE=`mktemp`; sudo wget "$URL" -qO $FILE && sudo gdebi $FILE --non-interactive; rm $FILE | |
fi | |
echo "start R and install commonly used packages" | |
# http://stackoverflow.com/q/4090169/1036500 | |
# Make an R script file to use in a moment... | |
LOADSTUFF="options(repos=structure(c(CRAN='http://cran.rstudio.com/'))) | |
update.packages(checkBuilt = TRUE, ask = FALSE) | |
packages <- c('codetools', 'Rcpp', 'devtools', 'knitr', 'ggplot2', 'plyr', 'dplyr', 'XML', 'RCurl', 'readxl', 'tidyr') | |
# just some of my most often used ones | |
install.packages(packages) | |
# and some from github | |
devtools::install_git(c('https://github.com/rstudio/rmarkdown.git', 'https://github.com/hadley/reshape'))" # close the R script | |
# put that R code into an R script file | |
FILENAME1="loadstuff.r" | |
sudo echo "$LOADSTUFF" > /tmp/$FILENAME1 | |
# Make a shell file that contains instructions in bash for running that R script file | |
# from the command line. There may be a simpler way, but nothing I tried worked. | |
NEWBASH='#!/usr/bin/env | |
sudo Rscript /tmp/loadstuff.r' | |
FILENAME2="loadstuff.sh" | |
# put that bash code into a shell script file | |
sudo echo "$NEWBASH" > /tmp/$FILENAME2 | |
# run the bash file to exectute the R code and install the packages | |
sh /tmp/loadstuff.sh | |
# clean up by deleting the temp file | |
rm /tmp/loadstuff.sh | |
# done. | |
echo "all done" | |
echo "type 'sudo rstudio' in the terminal to start RStudio" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# set cran repo so the user doesn't have to | |
options(repos=structure(c(CRAN='http://cran.rstudio.com/'))) | |
# update installed pacakges | |
update.packages(checkBuilt = TRUE, ask = FALSE) | |
# make a list of additional packages to install | |
packages <- c('codetools', 'Rcpp', 'devtools', 'knitr', 'ggplot2', 'plyr', 'dplyr', 'XML', 'RCurl', 'readxl', 'tidyr') | |
# install that list | |
install.packages(packages) | |
# and some from github | |
devtools::install_git(c('https://github.com/rstudio/rmarkdown.git', 'https://github.com/hadley/reshape'))" | |
message("All done installing additional packages") | |
# end | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
############# More detailed bits and pieces ############## | |
If archeos https://github.com/archeos/ArcheOS gets things right it could replace all of this, just run that in the VM | |
The Berkeley Computational Enviroment http://collaboratool.berkeley.edu/ is similar but a bit heavy | |
## First download a Linux distribution: | |
# Debian internet-installer image file: http://www.debian.org/distrib/netinst | |
# or | |
# Lubuntu, if looking for something smaller and using less memory, use the desktop cd from http://lubuntu.net/ | |
# then download and install Oracle VM VirtualBox: https://www.virtualbox.org/wiki/Downloads | |
## start VirtualBox | |
# Make new VM, follow prompts, set "Type: Linux" and "Version: Debian" or if using Lubuntu, "Version: Ubuntu" then ok | |
# at 'select start-up disk' choose the downloaded iso to make the VM from, then if using Debian, use the Debian graphic installer | |
# many fields can be left blank, try them and see, but DO be sure to select “Debian desktop environment” | |
# There are many useful guides for this (stop following when you get to the desktop, don't run any of the code in these how-tos) | |
# Here's a selection of how-tos (you might also search youtube) | |
# Mac: http://williamjturkel.net/2013/05/31/installing-debian-linux-in-a-vm/ | |
# http://www.brianlinkletter.com/installing-debian-linux-in-a-virtualbox-virtual-machine/ | |
# Windows: https://jtreminio.com/2012/07/setting-up-a-debian-vm-step-by-step/ | |
## install guest additions (GA) to allow for copy and pasting between VM and host | |
# http://askubuntu.com/questions/22743/how-do-i-install-guest-additions-in-a-virtualbox-vm | |
# first open terminal and type (without #) | |
# sudo apt-get update | |
# sudo apt-get install make gcc | |
# then at the terminal, change to the GA directory and type: | |
# sudo ./VBoxLinuxAdditions.run | |
# then | |
# sudo reboot | |
# Once VM has finished installing and Linux is up and running on your screen, | |
# open a terminal to continue, in Debian do: | |
Applications -> Accessories ->Tterminal | |
# Or in Lubuntu do: | |
Button in lower left of screen -> Accessories -> LXTerminal | |
# see shell script below for getting things installed, read through it before running to | |
# know what to expect, then copy-paste the shell script below into the text editor (gedit, leafpad etc.) on your VM, | |
# save locally on your VM, then run the shell script from the terminal by | |
# typing the next line in the terminal (without the #): | |
# bash install_things.sh | |
## Other things we want to do with a VM: | |
# to share the entire Linux VM with others | |
# http://superuser.com/questions/235376/how-to-share-a-virtualbox-image | |
file -> export appliance | |
# to swap files between host OS and guest OS: https://www.virtualbox.org/manual/ch04.html | |
# and don't forget to add self to share folder: https://www.virtualbox.org/ticket/7951 | |
# Debian: getting in and out of su: | |
# in: su - | |
# out: su yourusername | |
# Docker | |
# this seems to be important to set variables and make Docker work on windows | |
# eval "$(docker-machine.exe env docker)" | |
# To do: | |
# add right-click or keyboard shortcut to 'open terminal' cf. http://askubuntu.com/a/207448/119774 http://www.cyberciti.biz/faq/linux-gnome-open-terminal-shell-prompt-here/ | |
# see http://cran.r-project.org/bin/linux/ubuntu/README | |
# sudo sh -c 'echo "deb http://cran.rstudio.com/bin/linux/debian wheezy-cran3/" >> /etc/apt/sources.list' # if Debian | |
sudo sh -c 'echo "deb http://cran.rstudio.com/bin/linux/ubuntu trusty/" >> /etc/apt/sources.list' # if Lubuntu | |
# sudo sh -c 'echo "deb http://cran.rstudio.com/ raring-backports main restricted universe" >> /etc/apt/sources.list' # if Lubuntu | |
# sudo easy_install rpy2 # if using python: now we can sudo ipython notebook & use R within: http://nbviewer.ipython.org/github/ipython/ipython/blob/3607712653c66d63e0d7f13f073bde8c0f209ba8/docs/examples/notebooks/rmagic_extension.ipynb#Rmagic-Functions-Extension | |
From https://github.com/ropensci/docker/blob/master/ropensci/Dockerfile | |
## Add CRAN binaries and update | |
RUN add-apt-repository -y ppa:marutter/c2d4u && apt-get update -qq && apt-get upgrade -qy | |
## From the Build-Depends of the Debian R package: 968 MB! | |
RUN apt-get install -y --no-install-recommends gcc g++ gfortran libblas-dev liblapack-dev tcl8.5-dev tk8.5-dev bison groff-base libncurses5-dev libreadline-dev debhelper texinfo libbz2-dev liblzma-dev libpcre3-dev xdg-utils zlib1g-dev libpng-dev libjpeg-dev libx11-dev libxt-dev x11proto-core-dev libpango1.0-dev libcairo2-dev libtiff5-dev xvfb xauth xfonts-base texlive-base texlive-latex-base texlive-generic-recommended texlive-fonts-recommended texlive-fonts-extra texlive-extra-utils texlive-latex-recommended texlive-latex-extra default-jdk mpack bash-completion | |
## rmarkdown-related dependencies: 146 MB | |
RUN apt-get install -y --no-install-recommends texlive-humanities lmodern texlive-fonts-recommended texlive-latex-extra imagemagick pandoc pandoc-citeproc | |
## R binaries: 149 MB | |
RUN apt-get install -y r-cran-dplyr r-cran-devtools r-cran-rcpp r-cran-xml r-cran-rjava r-cran-rmysql r-cran-rsqlite libxslt1-dev r-cran-rsqlite.extfuns r-cran-rmysql r-cran-data.table r-cran-testthat r-cran-knitr r-cran-ggplot2 r-cran-mgcv r-cran-mass r-cran-tidyr r-cran-shiny r-cran-monetdb.r libxslt1-dev libpq-dev | |
## Not avaiable on marutter/c2d4u | |
# r-cran-rstudioapi r-cran-microbenchmark r-cran-rpostgresql r-cran-roxygen2 | |
RUN echo 'options("repos"="http://cran.rstudio.com", encoding="UTF-8")' > /.Rprofile | |
RUN Rscript -e 'install.packages(c("RPostgreSQL", "roxygen2", "microbenchmark", "rstudioapi"))' | |
RUN Rscript -e 'source("http://bioconductor.org/biocLite.R"); biocLite("rhdf5", ask=FALSE); biocLite("BiocInstaller")' | |
RUN Rscript -e 'devtools::install_github(c("rstudio/rmarkdown", "rstudio/packrat", "hadley/bigrquery", "hadley/reshape"))' | |
RUN Rscript -e 'install.packages(c("SSOAP", "XMLSchema", "Sxslt"), repo="http://www.omegahat.org/R", type="source")' | |
RUN Rscript -e 'devtools::install_github("egonw/rrdf", subdir="rrdflibs", build_vignettes=FALSE); devtools::install_github("egonw/rrdf", subdir="rrdf", build_vignettes=FALSE)' | |
### | |
ubuntu mini, https://help.ubuntu.com/community/Installation/MinimalCD | |
http://ask.xmodulo.com/install-minimal-ubuntu-desktop.html | |
## if buggy, get the pre-built VM from | |
http://www.osboxes.org/lubuntu/ |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment