andersonrobot

#!/bin/sh 
 
set -e 
 
PACKAGES="git qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils virt-manager git realpath autoconf bison flex texinfo libtool libtool-bin libncurses5-dev gawk gperf stgit build-essential fakeroot libexpat1-dev python-dev xorriso mtools dosfstools"
ONIE_URL="https://github.com/opencomputeproject/onie.git" 

# needed to locate mkdosfs
export PATH=$PATH:/sbin:/usr/sbin

andersonrobot

Oh my zsh.

Install ZSH.

sudo apt install zsh-autosuggestions zsh-syntax-highlighting zsh

Install Oh my ZSH.

andersonrobot

PSA: Versions newer than 2021.08 will not build. Starting at 2021.11, build will fail with

make: *** No rule to make target 'conf/crosstool/gcc-4.9.2/uClibc-ng-1.0.38/crosstool.x86_64.config', needed by '/home/robin/onie-build/build/x-tools/x86_64-g4.9.2-lnx3.2.69-uClibc-ng-1.0.38/build/.config'.  Stop.

Download DUE (do not use the one from debian repos, it's v3.0 as of time of writing and too old) Create buildenv with DUE

andersonrobot

VMware Workstation Pro 16.x Serials

YA7RA-F6Y46-H889Z-LZMXZ-WF8UA
ZV7HR-4YX17-M80EP-JDMQG-PF0RF
UC3XK-8DD1J-089NP-MYPXT-QGU80
GV100-84W16-M85JP-WXZ7E-ZP2R6
YF5X2-8MW91-4888Y-DNWGC-W68TF
AY1XK-0NG5P-0855Y-K6ZXG-YK0T4

VMware Workstation Player 16.x Serials

andersonrobot

Enable Dual Stack (IPv4 and IPv6) OpenFabric Routing

this gist is part of this series

This assumes you are running Proxmox 8.2 and that the line source /etc/network/interfaces.d/* is at the end of the interfaces file (this is automatically added to both new and upgraded installations of Proxmox 8.2).

This changes the previous file design thanks to @NRGNet for the suggestions to move thunderbolt settings to a file in /etc/network/interfaces.d it makes the system much more reliable in general, more maintainable esp for folks using IPv4 on the private cluster network (i still recommend the use of the IPv6 FC00 network you will see in these docs)

This will result in an IPv4 and IPv6 routable mesh network that can survive any one node failure or any one cable failure. Alls the steps in this section must be performed on each node

andersonrobot

Enable OSPF Routing on Thunderbolt Mesh

This has been deprectaed

It is now superceded by Openfabric Routing see here

continue at your own peril, for reference only now.

Old Gist

This will result in an IPv4 routable mesh network that can survive any one node failure or any one cable failure. All the steps in this section must be performed on each node

andersonrobot

ProxMox Cluster - Soup-to-Nutz

aka what i did to get from nothing to done.

note: these are designed to be primarily a re-install guide for myself (writing things down helps me memorize the knowledge), as such don't take any of this on blind faith - some areas are well tested and the docs are very robust, some items, less so). YMMV

Purpose of Proxmox cluster project

Required Outomces of cluster project

andersonrobot

CEPH HA Setup

Note this should only be done once you are sure you have reliable TB mesh network.

this is because proxmox UI seems fragile wrt to changing underlying network after configuration of ceph.

All installation done via command line due to gui not understanding the mesh network

This setup doesn't attempt to seperate the ceph public network and ceph cluster network (not same as proxmox clutser network), The goal is to get an easy working setup.

this gist is part of this series

andersonrobot

"city","city_ascii","lat","lng","country","iso2","iso3","admin_name","capital","population","id"
"Malishevë","Malisheve","42.4822","20.7458","Kosovo","XK","XKS","Malishevë","admin","","1901597212"
"Prizren","Prizren","42.2139","20.7397","Kosovo","XK","XKS","Prizren","admin","","1901360309"
"Zubin Potok","Zubin Potok","42.9144","20.6897","Kosovo","XK","XKS","Zubin Potok","admin","","1901608808"
"Kamenicë","Kamenice","42.5781","21.5803","Kosovo","XK","XKS","Kamenicë","admin","","1901851592"
"Viti","Viti","42.3214","21.3583","Kosovo","XK","XKS","Viti","admin","","1901328795"
"Shtërpcë","Shterpce","42.2394","21.0272","Kosovo","XK","XKS","Shtërpcë","admin","","1901828239"
"Shtime","Shtime","42.4331","21.0397","Kosovo","XK","XKS","Shtime","admin","","1901598505"
"Vushtrri","Vushtrri","42.8231","20.9675","Kosovo","XK","XKS","Vushtrri","admin","","1901107642"
"Dragash","Dragash","42.0265","20.6533","Kosovo","XK","XKS","Dragash","admin","","1901112530"

andersonrobot

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>PayloadContent</key>
	<array>
		<dict>
			<key>IKEv2</key>
			<dict>
				<key>AuthName</key>