fabrics-coal.md

Setting up fabrics in coal

Prerequisites

You need pretty much the newest version of everything. Go update it all. I'll wait.

sdcadm installations and configuring

In the Global Zone:

sdcadm post-setup dev-headnode-prov
sdcadm post-setup common-external-nics
sdcadm experimental update-docker
sdcadm post-setup cloudapi
sdcadm experimental portolan

Configuring Fabrics

In the NAPI zone, create the underlay nic tag and network:

napictl create-nictag sdc_underlay
napictl create-network name=sdc_underlay subnet=10.88.88.0/24 \
    provision_start_ip=10.88.88.205 provision_end_ip=10.88.88.250 \
    nic_tag=sdc_underlay vlan_id=0
napictl create-networkpool sdc_nat \
    networks=$(napictl list-networks name=external | json 0.uuid)

In the GZ:

echo "{
    \"default_underlay_mtu\": 1500,
    \"default_overlay_mtu\": 1400,
    \"sdc_nat_pool\": \"$(sdc-napi /network_pools?name=sdc_nat | json -H 0.uuid)\",
    \"sdc_underlay_assignment\": \"manual\",
    \"sdc_underlay_tag\": \"sdc_underlay\"
}" > /var/tmp/fabrics.conf
sdcadm experimental fabrics -c /var/tmp/fabrics.conf

Creating default fabric VLAN and network

In the NAPI zone, create the default fabric VLAN and network (replace e6fcbc64-3f32-11e2-a144-bf78292e9628 with your user UUID):

export MYUSER=e6fcbc64-3f32-11e2-a144-bf78292e9628
napictl create-fabric-vlan $MYUSER \
    vlan_id=2 name=default
napictl create-fabric-network $MYUSER 2 \
    name=default subnet=192.168.128.0/22 \
    provision_start_ip=192.168.128.5 \
    provision_end_ip=192.168.131.250 \
    gateway=192.168.128.1 resolvers=8.8.8.8,8.8.4.4

In the NAPI zone, create a default fabric network for the admin user:

export MYUSER=$(json -f /opt/smartdc/napi/config.json ufdsAdminUuid)
napictl create-fabric-vlan $MYUSER \
    vlan_id=2 name=default
napictl create-fabric-network $MYUSER 2 \
    name=default subnet=192.168.128.0/22 \
    provision_start_ip=192.168.128.5 \
    provision_end_ip=192.168.131.250 \
    gateway=192.168.128.1 resolvers=8.8.8.8,8.8.4.4

CN setup (can skip if you don't have a CN)

In the GZ, setup your CN and assign the sdc_underlay nic tag to it:

sdc-server setup -s 564d4d2c-ddd0-7be7-40ae-bae473a1d53e \
    external_nic=00:0c:29:a1:d5:48 sdc_underlay_nic=00:0c:29:a1:d5:48

In the NAPI zone, provision an underlay nic for that server:

napictl provision-nic \
    $(napictl list-networks name=sdc_underlay | json 0.uuid) \
    belongs_to_uuid=564d4d2c-ddd0-7be7-40ae-bae473a1d53e \
    belongs_to_type=server \
    owner_uuid=$(json -f /opt/smartdc/napi/config.json ufdsAdminUuid) \
    underlay=true

Reboot the server.

HN setup

In the NAPI zone, provision an underlay nic for the headnode:

napictl update-nic 00:50:56:3d:a7:95 nic_tags_provided=external,sdc_underlay
napictl provision-nic \
    $(napictl list-networks name=sdc_underlay | json 0.uuid) \
    belongs_to_uuid=564d843f-7cc8-835e-33f0-2f2a151bdfb4 \
    belongs_to_type=server \
    owner_uuid=$(json -f /opt/smartdc/napi/config.json ufdsAdminUuid) \
    underlay=true

In the Global Zone, Setup the headnode with a boot-time networking file:

/usbkey/scripts/mount-usb.sh
sdc-login dhcpd /opt/smartdc/booter/bin/hn-netfile > /mnt/usbkey/boot/networking.json

Unmount the usbkey and reboot:

umount /mnt/usbkey
reboot

You should now be able to provision on both the HN and CN.