Skip to content

Instantly share code, notes, and snippets.

@nbari
Forked from jahewson/smartos-on-a-budget.sh
Created November 3, 2019 09:34
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nbari/665c8307db4e4c08f8c3a11143ca923a to your computer and use it in GitHub Desktop.
Save nbari/665c8307db4e4c08f8c3a11143ca923a to your computer and use it in GitHub Desktop.
Installing and Configuring SmartOS on a budget server (with a /29)
# Licensed under CC BY 3.0 http://creativecommons.org/licenses/by/3.0/
# Derived works must attribute https://gist.github.com/4492300 at the beginning, and the date.
##################################################################
Installing and Configuring SmartOS on a budget server (with a /29)
##################################################################
# if you find this gist useful, please star it
# please be aware that budget hosting companies usually cut corners somewhere,
# you have been warned!
# thanks to: jamesog, linuxprofessor, ryancnelson for help with routing
###############
1. Installation
###############
# log in to the Linux "Recovery System".
# download the latest SmartOS USB image
wget https://download.joyent.com/pub/iso/latest-USB.img.bz2
bunzip2 latest-USB.img.bz2
# note: SmartOS requires a USB key on your server
# find out its device name using:
fdisk -l
# on my system it is /dev/sdd
# write the image to the USB key (/dev/sdd)
# IMPORTANT /dev/sdd WILL BE ERASED
dd if=latest-USB.img of=/dev/sdd bs=1024
# now make the USB drive bootable
fdisk /dev/sdd
# Command (m for help): a
# Partition number (1-4): 1
# Command (m for help): w
reboot
# now request a remote console (e.g. LANA, RAC) from the budget hosting company, and log in via the
# Java applet (usually).
# use the robot to request a reboot of the server, and press DEL when the POST shows to enter the BIOS.
# your BIOS is set to boot from the network - do not change that, you need it to gain access to the
# recovery system in the future. Instead, change the order of the fallback local boot options. These
# are labelled "Hard Drive BBS Priorities" on my motherboard - you want to set this to boot from the
# USB key.
# (OPTIONAL) [if you want your zpool to only use some (but not all) of the available drives, then make
# a note of the order in which they are displayed on the POST screen, so that you know which drives are
# which during SmartOS install - the will be in the same order]
# save settings and exit the BIOS
# machine will try network boot and fail, then try a local boot from the USB key. You should see the
# SmartOS GRUB screen now. Let it boot the SmartOS installer.
# follow the SmartOS install wizard, using 'dhcp' as the IP address. Reboot, and you're finished with
# the LANA.
###
# MESSED UP? If it all goes wrong, you can boot SmartOS with the (noinstall) option, using the image's
# default root password. Then list disks with 'format' and delete them with 'fdisk /dev/rdsk/c0t0d0p0'
# - note the p0 at the end. Root password available here https://download.joyent.com/pub/iso/
#################################
2. Basic Configuration (OPTIONAL)
#################################
# (OPTIONAL) set a hostname: http://wiki.smartos.org/display/DOC/Setting+a+static+hostname+at+boot+time
# (OPTIONAL) upload a root SSH key: http://www.perkin.org.uk/posts/smartos-global-zone-tweaks.html
###########################
3. Configuring a /29 subnet
###########################
# Many budget hosting companies will give you a /29 (or indeed a /28) subnet which is *statically routed*
# to your server's main IP. In the subnet x.x.x.200/29 the first address (x.x.x.200) is used to identify
# the network, and the last address (x.x.x.207) is used for broadcast, leaving six usable IP addresses
# (but we have to use one for the gateway, so we only get FIVE usable IPs).
# The budget hosting company will route the subnet traffic to the main IP of your server, and expect you
# to provde your own gateway for the subnet. Threfore we have to set up a vnic in the global zone to act as
# a router for the subnet. This uses up the the first available ip of our /29.
dladm create-vnic -l rge0 vnic0 # rge0 = physical nic (from ifconfig)
ifconfig vnic0 plumb x.x.x.201 netmask 255.255.255.248 up # x.x.x.201 = first usable ip
svcadm enable route # turn on ipv4 routing
# check that you can now ping x.x.x.201 from the internet
# now we can launch zones using the five remaining ips, for example x.x.x.202
# the gateway is set to x.x.x.201 which is the router we just set up in the global zone
cat > /tmp/zonedef << EOF
{
"brand": "joyent",
"autoboot": true,
"dataset_uuid": "fdea06b0-3f24-11e2-ac50-0b645575ce9d",
"nics": [
{
"nic_tag": "admin",
"ip": "x.x.x.202",
"netmask": "255.255.255.248",
"gateway": "x.x.x.201"
}
]
}
EOF
vmadm create -f /tmp/zonedef
# check that you can now ping x.x.x.202 from the internet
# and that the zone can reach the internet:
zlogin <Zone UUID>
ping google.com
# google.com is alive
exit
### Persistance ###
# if everything works, then we need to persist the configuration of the global zone so that it
# survives a reboot:
# first create an SMF service to run a script on boot
# you don't need to customise any of this XML
mkdir -p /opt/custom/smf
cat >> /opt/custom/smf/subnet-routing-setup.xml << EOF
<!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>
<service_bundle type='manifest' name='export'>
<service name='site/subnet-routing-setup' type='service' version='1'>
<create_default_instance enabled='true'/>
<single_instance/>
<dependency name='network' grouping='require_all' restart_on='error' type='service'>
<service_fmri value='svc:/milestone/network:default'/>
</dependency>
<dependency name='filesystem' grouping='require_all' restart_on='error' type='service'>
<service_fmri value='svc:/system/filesystem/local'/>
</dependency>
<exec_method name='start' type='method' exec='/opt/custom/scripts/subnet-routing-setup' timeout_seconds='60'>
<method_context>
<method_credential user='root' group='staff'/>
<method_environment>
<envvar name='PATH' value='/usr/bin:/usr/sbin:/bin'/>
</method_environment>
</method_context>
</exec_method>
<exec_method name='stop' type='method' exec=':true' timeout_seconds='0'/>
<property_group name='startd' type='framework'>
<propval name='duration' type='astring' value='transient'/>
</property_group>
</service>
</service_bundle>
EOF
# then create the actual script
# you need to customise this as shown previously
mkdir -p /opt/custom/scripts/
cat >> /opt/custom/scripts/subnet-routing-setup << EOF
#!/bin/sh
. /lib/svc/share/smf_include.sh
dladm create-vnic -l rge0 vnic0 # <-- customise
ifconfig vnic0 plumb x.x.x.201 netmask 255.255.255.248 up # <-- customise
svcadm enable route
exit $SMF_EXIT_OK
EOF
chmod +x /opt/custom/scripts/subnet-routing-setup
# now, reboot and check that everything works
# you can use ifconfig, svcs, and routeadm to debug things.
reboot
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment