scyto

Enable & Using vGPU Passthrough

This gist is almost entirely not unlike Derek Seaman's awesome blog:

Proxmox VE 8: Windows 11 vGPU (VT-d) Passthrough with Intel Alder Lake

As such please refer to that for pictures, here i will capture the command lines I used as i sequence the commands a little differently so it makes more logic to me.

This gists assumes you are not running ZFS and are not passing any other PCIE devices (as both of these can require addtional steps - see Derek's blog for more info)

This gist assumes you are not running proxmox in UEFI Secure boot - if you are please refer entirely to dereks blog.

scyto

Description

This template runs my experimental NPM service as part of evaluating if i should stop using my hand crafted nginx reverse proxy. Note change your passwords and don't use the examples bellow :-) )

State Considerations for SWARM

This container has a database, as such I store the data on the gluster volume. I am currenlty using a mount point to store the voume data on glusterfs. I restrict to 1 instance of each container to avoid database corruption from having two instances. Change example usernames and passwords as needed. leave hostname as db (name resolution works fine using the method)

scyto

Proxmox Backup

At this time proxmox backup only backs up VM and Containers - ths guide covers that.

What i didn't realize is the backup job is still defined on the cluster and PBS provides a new storage type that dedupes and managed all the vzdump files created - which is cool.

I decided to run proxmox backup on my Synology NAS where it has more reliable connection to the NAS (i.e. via memory) for doing deduple, garbage collection, prune, verification etc. However the steps here generally remain true.

Once again i used one of Derek Seaman's Awesome Blogs for the basis of this - but with my own tweaks (like using SMB instead of CIFS. As of 9/21 my tweaks are signifcnant, in the original blog it is missing steps to enable encoding acceleration in CTs and VMs.

this gist is part of this series

scyto

This (and related gists) captures how i created my docker swarm architecture. This is intended mostly for my own notes incase i need to re-creeate anything later! As such expect some typos and possibly even an error...

Installation Step-by-Step

Each major task has its own gist, this is to help with maitainability long term.

1. Install Debian VM for each docker host
2. install Docker
3. Configure Docker Swarm
4. Install Portainer
5. Install KeepaliveD
6. glusterFS disk prep, install & config

scyto

ProxMox Cluster - Soup-to-Nutz

aka what i did to get from nothing to done.

note: these are designed to be primarily a re-install guide for myself (writing things down helps me memorize the knowledge), as such don't take any of this on blind faith - some areas are well tested and the docs are very robust, some items, less so). YMMV

Purpose of Proxmox cluster project

Required Outomces of cluster project

scyto

Azure Active Directory (AAD) Auth

This gist assumes a working Azure AD (not Azure AD-DS is already up and fully configured) This gist assumes working DNS / name resolution on your internal network.

this gist is part of this series

Create App Registrations

All of these steps will be done in the Azure Portal AAD UI

1. Select App Registration from the nav bar

scyto

Thunderbolt Networking

this gist is part of this series

NOTE FOR THIS TO BE RELIABLE ON NODE RESTARTS YOU WILL NEED PROXMOX KERNEL 6.2.16-14-pve OR HIGER

This fixes issues i bugged with the thunderbolt / thunderbolt-net maintainers (i will take everyones thanks now, lol)

Install LLDP - this is great to see what nodes can see which.

• install lldpctl with apt install lldpd

scyto

Enable IPv6 OSPF Routing on Thunderbolt-Net Mesh

This requires proxmox kernel 6.2.16-14-pve or higher due to bugs in earlier version.s

This will result in a routable mesh network that can survive any one node failure or any one cable failure. Alls the steps in this section must be performed on each node

IPv6 OSPF connectivity over TB cluster network

Enable IPv6 forwarding

Using IPv6 to take advantage of not needing to use addresses - does make things simpler

scyto

Enable Dual Stack (IPv4 and IPv6) OpenFabric Routing

This will result in an IPv4 and IPv6 routable mesh network that can survive any one node failure or any one cable failure. Alls the steps in this section must be performed on each node

Note for ceph do not dual stack - either use IPv4 or IPv6 addressees for all the monitors, MDS and daemons - despite the docs implying it is ok my findings on quincy are is it is funky....

this gist is part of this series

Create Loopback interfaces

Doing this means we don't have to give each thunderbolt a manual IPv6 or IPv4 addrees and that these addresses stay constant no matter what.

scyto

Configuring CephFS to store ISOs for VMs and Templates for Containers

This gist assume you have setup Ceph as per earlier gist in this sequence

I found it annoying to upload ISO to each node in the cluster, download CT templates to each node.
I thought setting up cedph for these would be a great idea (and it is).

this gist is part of this series

create cephfs pools

1. In the gui navigate to Datcenter > node name > ceph