Skip to content

Instantly share code, notes, and snippets.

@sax
Last active August 29, 2015 13:56
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save sax/9138190 to your computer and use it in GitHub Desktop.
Download latest SmartOS platform image and turn it into a VirtualBox VM

UPDATE: this has been made slightly more official by way of turning it into a repo: https://github.com/sax/vagrant-smartos-packager

Notes:

  • This will create files on the local disk
  • You'll probably want to create and cd into a working directory before doing any of this
  • This is very much a work in progress. This gist is to capture lots of tiny changes in code, without managing the horrible git history that would ensue in a real git repo. Once this works to completion, it's going into a github repo.

On local machine, download the latest SmartOS platform image and turn it into a virtualbox image:

curl -k https://gist.githubusercontent.com/sax/9138190/raw/mksmartvm | bash -s

When that spins up, set the network configuration and zone information as the defaults. When the prompt asks to create a root password, set it to vagrant.

Now log in as root, and run the following:

curl -k https://gist.githubusercontent.com/sax/9138190/raw/prepare_global_zone

This will install a service that allows you to create and manage users in the global zone, with info persisted in /usbkey. See the SmartOS wiki for more info.

Reboot the zone to ensure that the services added in the last step are loaded.

Now create a vagrant user:

curl -k https://gist.githubusercontent.com/sax/9138190/raw/prepare_gz_users

The vagrant user will be created with password vagrant, root privileges (Primary Administrator) and a small profile with sudo aliased to pfexec.

Now shut down the VM using a graceful shutdown signal.

At this point you can package up the VM into a vagrant box.

vagrant package --base SmartOS-20140221T042147Z --output SmartOS-20140221T042147Z.box
vagrant box add SmartOS-20140221T042147Z SmartOS-20140221T042147Z.box
#!/usr/bin/bash
userfiles=( /etc/passwd /etc/shadow /etc/group /etc/ouser_attr /etc/user_attr \
/etc/security/policy.conf /etc/security/auth_attr \
/etc/security/exec_attr /etc/security/prof_attr )
case "$1" in
'start')
if [[ -n $(/bin/bootparams | grep '^smartos=true') ]]; then
touch ${userfiles[@]}
sleep 1
for file in ${userfiles[*]}; do
ukf=/usbkey/$(basename ${file})
test -e $ukf && touch $ukf
done
if [[ -e /usbkey/user_attr ]]; then
cp /usbkey/user_attr /etc/user_attr
fi
fi
;;
esac
<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='manifest' name='export'>
<service name='site/install-userfiles' type='service' version='0'>
<create_default_instance enabled='true'/>
<single_instance/>
<dependency name='fs-local' grouping='require_all' restart_on='error' type='service'>
<service_fmri value='svc:/system/filesystem/local'/>
</dependency>
<dependency name='fs-root' grouping='require_all' restart_on='error' type='service'>
<service_fmri value='svc:/system/filesystem/root'/>
</dependency>
<method_context/>
<exec_method name='start' type='method' exec='/opt/custom/method/install-userfiles start' timeout_seconds='60'/>
<exec_method name='stop' type='method' exec=':true' timeout_seconds='60'/>
<property_group name='startd' type='framework'>
<propval name='duration' type='astring' value='transient'/>
<propval name='ignore_error' type='astring' value='core,signal'/>
</property_group>
<property_group name='application' type='application'/>
<stability value='Evolving'/>
<template>
<common_name>
<loctext xml:lang='C'>Mount user and RBAC data from /usbkey</loctext>
</common_name>
</template>
</service>
</service_bundle>
#!/bin/bash
#
# Configurables:
#
# - Disk size is in GB
# - Memory size is in MB
# - SSH port is the local forwarded port to the VM:22
#
# props: http://www.perkin.org.uk/posts/automated-virtualbox-smartos-installs.html
usage() {
echo "$0 [options]"
echo " -d --disksize Size of disk in GB (default 32)"
echo " -m --memsize Memory to allocate in MB (default 1024)"
echo " -p --sshport SSH port to forward (default 2222)"
}
DISKSIZE="32"
MEMSIZE="1024"
SSHPORT="2222"
while getopts ":d:m:p:" arg; do
case "$arg" in
d) DISKSIZE=${OPTARG} ;;
m) MEMSIZE=${OPTARG} ;;
p) SSHPORT=${OPTARG} ;;
*) usage; exit;;
esac
done
latest_path=$(curl "https://us-east.manta.joyent.com/Joyent_Dev/public/SmartOS/latest")
dlsite="https://us-east.manta.joyent.com${latest_path}"
vboxdir=$(VBoxManage list systemproperties \
| awk '/^Default.machine.folder/ { print $4 }')
#
# Find a suitable md5sum program.
#
if type md5 >/dev/null 2>&1; then
md5sum='md5'
column='NF'
elif type digest >/dev/null 2>&1 &&
digest md5 /dev/null >/dev/null 2>&1; then
md5sum='digest md5'
column='NF'
elif type digest >/dev/null 2>&1 &&
digest -a md5 /dev/null >/dev/null 2>&1; then
md5sum='digest -a md5'
column='1'
elif type md5sum >/dev/null 2>&1; then
md5sum='md5sum'
column='1'
elif type openssl >/dev/null 2>&1 &&
openssl md5 -hex /dev/null >/dev/null 2>&1; then
md5sum='openssl md5 -hex'
column='NF'
else
echo "ERROR: Sorry, could not find an md5 program" 1>&2
exit 1
fi
#
# Download MD5 file and parse it for the latest ISO image and checksum
#
curl -o smartos-sums.txt ${dlsite}/md5sums.txt 2>/dev/null
latest_md5=$(awk '/\.iso/ { print $1 }' smartos-sums.txt)
smartos_version=$(sed -ne "/^${latest_md5}/s/.*-\(.*\).iso/\1/p" \
smartos-sums.txt)
if [ -z "${smartos_version}" ]; then
echo "ERROR: Couldn't determine latest version"
exit 1
fi
vmname="SmartOS-${smartos_version}"
#
# Download the latest ISO image and verify
#
mkdir -p "${vboxdir}/${vmname}"
if [ ! -f "${vboxdir}/${vmname}/smartos-${smartos_version}.iso" ]; then
echo "Downloading ${dlsite}/smartos-${smartos_version}.iso"
curl -o "${vboxdir}/${vmname}/smartos-${smartos_version}.iso" \
${dlsite}/smartos-${smartos_version}.iso
dl_md5=$(${md5sum} "${vboxdir}/${vmname}/smartos-${smartos_version}.iso" \
| awk '{ print $'${column}' }')
if [ -z "${dl_md5}" ]; then
echo "ERROR: Couldn't fetch ISO image"
exit 1
fi
if [ "${latest_md5}" != "${dl_md5}" ]; then
echo "ERROR: md5 checksums do not match"
exit 1
fi
fi
#
# Create VirtualBox VM
#
echo "Creating/Updating Virtual Machine"
VBoxManage showvminfo "${vmname}" >/dev/null 2>&1
if [ $? -eq 0 ]; then
# VM already exists, just update the ISO image
VBoxManage storageattach "${vmname}" --storagectl "IDE Controller" \
--port 1 --device 0 --type dvddrive \
--medium "${vboxdir}/${vmname}/smartos-${smartos_version}.iso"
else
# Create the VM
VBoxManage createvm --name "${vmname}" --ostype OpenSolaris_64 --register
VBoxManage storagectl "${vmname}" --name "IDE Controller" --add ide
# Attach the ISO image
VBoxManage storageattach "${vmname}" --storagectl "IDE Controller" \
--port 1 --device 0 --type dvddrive \
--medium "${vboxdir}/${vmname}/smartos-${smartos_version}.iso"
# Create and attach the zone disk
VBoxManage createhd --filename "${vboxdir}/${vmname}/smartos-zones.vdi" \
--size $(echo "${DISKSIZE}*1024" | bc)
VBoxManage storageattach "${vmname}" --storagectl "IDE Controller" \
--port 0 --device 0 --type hdd \
--medium "${vboxdir}/${vmname}/smartos-zones.vdi"
# Set misc settings
VBoxManage modifyvm "${vmname}" --boot1 dvd --boot2 disk --boot3 none
VBoxManage modifyvm "${vmname}" --memory ${MEMSIZE}
VBoxManage modifyvm "${vmname}" --natpf1 "SSH,tcp,,${SSHPORT},,22"
# VBoxManage modifyvm "${vmname}" --nic1 bridged \
# --bridgeadapter1 `ifconfig | awk -F: '/^en/ { print $1 }' | head -1` \
# --nicpromisc1 allow-all
fi
#
# Start it up
#
echo "Starting Virtual Machine"
VirtualBox --startvm "${vmname}" &
#!/usr/bin/bash
# See http://wiki.smartos.org/display/DOC/Persistent+Users+and+RBAC+in+the+Global+Zone
persistent_files=( /etc/passwd /etc/shadow /etc/group /etc/ouser_attr /etc/user_attr \
/etc/security/policy.conf /etc/security/auth_attr \
/etc/security/exec_attr /etc/security/prof_attr )
ukeystor="/usbkey"
case "$1" in
'start')
if [[ -n $(/bin/bootparams | grep '^smartos=true') ]]; then
for file in ${persistent_files[*]}; do
ukf=${ukeystor}/$(basename $file)
if [[ -z $(/usr/sbin/mount -p | grep $file) ]]; then
if [[ $file -ot $ukf ]]; then
cp $ukf $file
echo "stor->sys: $file"
else
cp $file $ukf
echo "sys->stor: $file"
fi
touch $file $ukf
mount -F lofs $ukf $file
fi
done
fi
;;
'stop')
for file in ${persistent_files[*]}; do
if [[ -n $(/usr/sbin/mount -p | grep $file) ]]; then
umount $file && touch $file
fi
done
;;
*)
echo "Usage: $0 { start | stop }"
echo " When disabled, users can be modified in the SmartOS global zone"
echo " When enabled, users can not be modified"
exit 1
;;
esac
<?xml version='1.0'?>
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='manifest' name='export'>
<service name='site/persist-userfiles' type='service' version='0'>
<create_default_instance enabled='true'/>
<single_instance/>
<dependency name='filesystem' grouping='require_all' restart_on='error' type='service'>
<service_fmri value='svc:/system/filesystem/local'/>
</dependency>
<dependency name='userfiles' grouping='require_all' restart_on='error' type='service'>
<service_fmri value='svc:/site/install-userfiles'/>
</dependency>
<method_context/>
<exec_method name='start' type='method' exec='/opt/custom/method/persist-userfiles start' timeout_seconds='60'/>
<exec_method name='stop' type='method' exec='/opt/custom/method/persist-userfiles stop' timeout_seconds='60'/>
<property_group name='startd' type='framework'>
<propval name='duration' type='astring' value='transient'/>
<propval name='ignore_error' type='astring' value='core,signal'/>
</property_group>
<property_group name='application' type='application'/>
<stability value='Evolving'/>
<template>
<common_name>
<loctext xml:lang='C'>Mount user and RBAC data from /usbkey</loctext>
</common_name>
</template>
</service>
</service_bundle>
# curl -k https://gist.githubusercontent.com/sax/9138190/raw/prepare_global_zone | bash -s
echo "*************************************************************"
echo "Updating system to allow user modification"
mkdir -p /opt/custom/smf
mkdir -p /opt/custom/method
curl --silent -k https://gist.githubusercontent.com/sax/9138190/raw/install-userfiles -o /opt/custom/method/install-userfiles
curl --silent -k https://gist.githubusercontent.com/sax/9138190/raw/install-userfiles.xml -o /opt/custom/smf/install-userfiles.xml
curl --silent -k https://gist.githubusercontent.com/sax/9138190/raw/persist-userfiles -o /opt/custom/method/persist-userfiles
curl --silent -k https://gist.githubusercontent.com/sax/9138190/raw/persist-userfiles.xml -o /opt/custom/smf/persist-userfiles.xml
chmod -R +x /opt/custom/method
echo "*************************************************************"
echo "In order to use tools such as usermod, please do the following:"
echo " svcadm disable -s persist-userfiles"
echo " useradd ..."
echo " svcadm enable -s persist-userfiles"
echo
echo
echo "*************************************************************"
echo "Now reboot to ensure that everything is kosher."
echo "*************************************************************"
echo "Switching system into user modifiable state"
svcadm disable -s persist-userfiles
echo "*************************************************************"
echo "Creating vagrant user"
useradd -d /usbkey/vagrant -m -s /bin/bash vagrant
usermod -P'Primary Administrator' vagrant
echo "*************************************************************"
echo "Setting password for Vagrant user. Please set it to 'vagrant'"
passwd vagrant
echo "*************************************************************"
echo "Setting password for root user. Please set it to 'vagrant'"
passwd root
echo "*************************************************************"
echo "Switching system back into non-user modifiable state"
svcadm enable -s persist-userfiles
echo "*************************************************************"
echo "Configuring profile for vagrant user"
echo 'if [ -e $HOME/.bashrc ]; then' > /usbkey/vagrant/.bash_profile
echo ' . $HOME/.bashrc' >> /usbkey/vagrant/.bash_profile
echo 'fi' >> /usbkey/vagrant/.bash_profile
echo 'export PATH=/usr/bin:/usr/sbin/:smartdc/bin:/opt/local/bin:/opt/local/sbin:/usbkey/vagrant/bin' >> /usbkey/vagrant/.bash_profile
chown vagrant:other /usbkey/vagrant/.bash_profile
echo "*************************************************************"
echo "Configuring ssh access for vagrant user"
mkdir -p /usbkey/vagrant/.ssh
echo "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key" > /usbkey/vagrant/.ssh/authorized_keys
echo 'export PATH=/usr/bin:/usr/sbin/:smartdc/bin:/opt/local/bin:/opt/local/sbin:/usbkey/vagrant/bin' > /usbkey/vagrant/.ssh/environment
chown -R vagrant:other /usbkey/vagrant/.ssh
chmod -R 0700 /usbkey/vagrant/.ssh
mkdir -p /usbkey/vagrant/bin
echo '#!/usr/bin/bash' > /usbkey/vagrant/bin/sudo
echo 'pfexec $@' >> /usbkey/vagrant/bin/sudo
chmod -R 0777 /usbkey/vagrant/bin
# -*- mode: ruby -*-
# vi: set ft=ruby :
VAGRANTFILE_API_VERSION = "2"
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = "SmartOS-20140221T042147Z"
config.vm.synced_folder ".", "/vagrant", type: "nfs"
end
@sax
Copy link
Author

sax commented Feb 27, 2014

current status:

  • no guest additions
  • can't find sudo
  • no shared folders
  • need to configure vagrant user's ssh environment for PATH?
  • networking for non-global zones is almost certainly screwed
    • once we can spin up zones using a vagrant plugin, zone networking can be solved. doesn't really make sense to spend time on now.

Probably need to configure shares in Vagrantfile to use NFS, like how everyone does vagrant with FreeBSD.

ideas:

  • drop in sudo as a bash script wrapper for pfexec in /usbkey/vagrant/bin
  • use /usbkey/vagrant/.ssh/environment to set PATH, include /usbkey/vagrant/bin
Bringing machine 'default' up with 'virtualbox' provider...
[default] Importing base box 'SmartOS-20140221T042147Z'...
[default] Matching MAC address for NAT networking...
[default] Setting the name of the VM...
[default] Clearing any previously set forwarded ports...
[default] Clearing any previously set network interfaces...
[default] Preparing network interfaces based on configuration...
[default] Forwarding ports...
[default] -- 22 => 2222 (adapter 1)
[default] Booting VM...
[default] Waiting for machine to boot. This may take a few minutes...
[default] Machine booted and ready!
[default] No guest additions were detected on the base box for this VM! Guest
additions are required for forwarded ports, shared folders, host only
networking, and more. If SSH fails on this machine, please install
the guest additions and repackage the box to continue.

This is not an error message; everything may continue to work properly,
in which case you may ignore this message.
[default] Mounting shared folders...
[default] -- /vagrant
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!

sudo mkdir -p /vagrant

Stdout from the command:



Stderr from the command:

bash: line 2: sudo: command not found

@sax
Copy link
Author

sax commented Feb 28, 2014

Argh. Since /etc is a RAM disk loaded at boot, the timestamp on all files is boot time. Depending on when ensure-userfile-precedence runs, /etc/file might not be -ot than /usbkey/file.

Perhaps it should be reversed and -nt, so on system boot usbkey files always take precedence?

@sax
Copy link
Author

sax commented Feb 28, 2014

Updated to ensure that /etc/user_attr is really persisted between reboots (hopefully. that race condition completely sucks and probably will come back when least expected).

  • added a fake sudo command in /usbkey/vagrant/bin
  • hard code a PATH for vagrant user

Now we're at this error:

[default] No guest additions were detected on the base box for this VM! Guest
additions are required for forwarded ports, shared folders, host only
networking, and more. If SSH fails on this machine, please install
the guest additions and repackage the box to continue.

This is not an error message; everything may continue to work properly,
in which case you may ignore this message.
NFS requires a host-only network to be created.
Please add a host-only network to the machine (with either DHCP or a
static IP) for NFS to work.

@sax
Copy link
Author

sax commented Feb 28, 2014

Tries to alter /etc/exports, which doesn't exist in SmartOS. Haven't looked further, yet.

@bixu
Copy link

bixu commented Feb 28, 2014

FYI, any zfs filesystem on Illumos can be exported over NFS by setting an attribute like so:

zfs set sharenfs=on

This could potentially be your SmartOS zone...

@sax
Copy link
Author

sax commented Feb 28, 2014

@bixu yeah. My understanding is that the host (OS X in this case) creates the share, then the guest (SmartOS GZ) mounts that share. Not sure why it needs to update /etc/exports.

There's also this: hashicorp/vagrant#2560, though my brain is running on empty. Need to get more sleep before diving deep into this.

Hmmmmmm this would be a great place to plug in autofs, if it works in the global zone and if I can hook it into vagrant.

@sax
Copy link
Author

sax commented Feb 28, 2014

@bixu also: long term plan is to have vagrant spin up global zone, then use a vagrant plugin to install and connect to local zones. Probably doesn't actually need to mount anything in the global zone, so in the short term it may also be better to try to skip this step.

possibly using http://rubygems.org/gems/vagrant-smartos, or at least something inspired by and heavily drawn from it.

@sax
Copy link
Author

sax commented Mar 6, 2014

Ahhhh /etc/exports is local to the Mac.

@sax
Copy link
Author

sax commented Mar 6, 2014

https://github.com/sax/vagrant

Pending tests, but succeeds as a proof of concept. Once I can write tests this can become a pull request.

@sax
Copy link
Author

sax commented Mar 6, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment