Download, configure and run Cloudflare tunnels for Start9 services

cloudflare_tunnels.sh

#!/usr/bin/env bash

# Warn and prompt
read -p "Note, this script is reckless! You should not be exposing your Start9 the 
Internet like this. This will allow Cloudflare to read all processed data.
Do you really want to continue?? [Y/N]" -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]; then

# Variables
TUNNEL_NAME=start9
TUNNEL_CONFIG=${HOME}/${TUNNEL_NAME}_tunnel.yml
CLOUDFLARED_DIR=${HOME}/.cloudflared
BOLD=$(tput bold) # Start bold text
NORMAL=$(tput sgr0) # Stop bold text; turn off all attributes

# Place each domain.name_cert.pem in $HOME, example:
#
# $ ls $HOME
# domain1.com_cert.pem  domain2.com_cert.pem
DOMAINS=('domain1.com' 'domain2.com')

# SERVICES_FILE is a csv file, example:
#
# services.csv
# -----
# service,public_hostname,protocol,port
# btcpayserver,btcpay.domain1.com,http,
# mempool,mempool.domain2.com,http,8080
# nostr,nostr.domain1.com,http,8080
# nostr,nostr.domain2.com,http,8080
SERVICES_FILE=${HOME}/services.csv

# download and install cloudflared
curl -L --output cloudflared.deb https://github.com/cloudflare/cloudflared/releases/latest/download/cloudflared-linux-amd64.deb && 
sudo dpkg -i cloudflared.deb 

# This section deals with stopping cloudflared and removing installed services
## Stop cloudflared service
sudo systemctl stop cloudflared 2> /dev/null
## Remove cloudflared service
sudo cloudflared service uninstall 2> /dev/null
## Kill all clourdflared processes by id
CLOUDFLARED_PIDS=$(pidof cloudflared)
if [[ -n $CLOUDFLARED_PIDS ]]; then sudo kill $CLOUDFLARED_PIDS; fi

# Check if cert files exists for each domain
for DOMAIN in ${DOMAINS[@]}; do
    if [[ -e ${HOME}/${DOMAIN}_cert.pem ]]; then
        echo "INFO: ${HOME}/${DOMAIN}_cert.pem exists."
    else
        echo "${BOLD}WARN: ${HOME}/${DOMAIN}_cert.pem does not exist${NORMAL}. Running \`cloudflared tunnel login\`.."
        sleep 5
        rm -fv ${CLOUDFLARED_DIR}/cert.pem
        cloudflared tunnel login
        echo "INFO: Copying ${CLOUDFLARED_DIR}/cert.pem to ${HOME}/${DOMAIN}_cert.pem"
        cp -fv ${CLOUDFLARED_DIR}/cert.pem ${HOME}/${DOMAIN}_cert.pem
    fi

    ## List and delete existing tunnels
    echo "INFO: List and delete existing tunnels..."
    EXISTING_TUNNELS=$(cloudflared tunnel --origincert ${HOME}/${DOMAIN}_cert.pem list | grep -ve 'You\|CREATED' | awk '{print $1}' | xargs)
    for TUNNEL in $EXISTING_TUNNELS; do
        cloudflared tunnel --origincert ${HOME}/${DOMAIN}_cert.pem info $TUNNEL
        cloudflared tunnel --origincert ${HOME}/${DOMAIN}_cert.pem delete $TUNNEL
    done
done

# Create tunnel and set TUNNEL_ID
TUNNEL_ID=$(cloudflared tunnel --origincert ${HOME}/${DOMAINS[0]}_cert.pem create ${TUNNEL_NAME} | grep Created | awk '{print $NF}')

# Begin creating tunnel config
echo "tunnel: ${TUNNEL_ID}" > ${TUNNEL_CONFIG}
echo -e "credentials-file: ${HOME}/${TUNNEL_ID}.json\n\ningress:" >> ${TUNNEL_CONFIG}

# Read the SERVICES_FILE and ignore the first line
{
    read
    while IFS=, read -r SERVICE PUBLIC_HOSTNAME PROTOCOL PORT; do

        # Set SERVICE_ADDRESS according to empty/non-empty PORT
        if [[ -n ${PORT} ]]; then
            SERVICE_ADDRESS="${PROTOCOL}://${SERVICE}.embassy:${PORT}"
        else
            SERVICE_ADDRESS="${PROTOCOL}://${SERVICE}.embassy"
        fi
        
        # Write ingress rule entries to tunnels config
        echo "  - hostname: ${PUBLIC_HOSTNAME}" >> ${TUNNEL_CONFIG}
        echo "    service: ${SERVICE_ADDRESS}" >> ${TUNNEL_CONFIG}

        # Add PUBLIC_HOSTNAME to tunnel routes using the correct domain certificate
        for DOMAIN in ${DOMAINS[@]}; do
            if [[ ${PUBLIC_HOSTNAME} == *"${DOMAIN}"* ]]; then
                echo "INFO: Adding ${PUBLIC_HOSTNAME} to tunnel routes..."
                cloudflared tunnel --origincert ${HOME}/${DOMAIN}_cert.pem route dns -f ${TUNNEL_ID} $PUBLIC_HOSTNAME
            fi
        done
                
    done
} < $SERVICES_FILE

# Finish writing tunnel config
echo "  - service: http_status:404" >> ${TUNNEL_CONFIG}

# Delete possible conflicting configuration
sudo rm -fv /etc/cloudflared/config.yml
# Install cloudflared service to systemctl
sudo cloudflared --config ${TUNNEL_CONFIG} -f service install

# Restart cloudflared service and display the status
sudo systemctl restart cloudflared.service 
sudo systemctl status cloudflared.service

fi

⚠️ DON'T USE THIS ⚠️

Note, this script is reckless! You should not be exposing your Start9 the
Internet like this. This will allow Cloudflare to read all processed data.

Background

I created this for myself and I really hope I didn't spend hours on something that someone has already done and shared (I guess I should have searched it a bit more), but either way I hope that someone will find this useful.

Instructions

1. Copy this file to your Start9
2. Make this file executable chmod +x cloudflare_tunnels.sh
3. Create a CSV file following the format described starting line 16
4. Modify the variables at the beginning of the file as necessary
5. Ensure you have entered your domain names in line 14. Note, this is a bash array and will only work when multiple domains are managed by the same Cloudflare account.

Caution

• Existing cloudflared tunnels are deleted. Based on what I've read online, the intended design is one tunnel per system.
• This uninstalls the cloudflared.service and reinstalls it after building a new config
• You may want to backup ~/.cloudflared, /root/.cloudflared, and /etc/cloudflared before using this script

Please provide any feedback you may have. Good luck!

Updates

2024-04-29 - Add prompt and warning
2023-11-14 - Suppress error message when attempting to stop and uninstall cloudflared
2023-11-14 - Use service hostname and remove docker command to get service IP address
2023-08-09 - Fixed failed kill output by adding a conditional
2023-07-07 - Fixed delete tunnels by moving it to the appropriate section

That worked perfectly, thanks a lot!

Just a few small things.

1. It seemed to be necessary to manually create the tunnel and install it in start9 initially. Otherwise, the script would fail.
2. At first, I wasn't aware that the *_cert.pem file was the certificate created by cloudflared for the tunnel. I thought it was the client cert or origin server cert (since I am not that experienced in this kind of stuff)

1. When your StartOS reboots, cloudflared and foreign files are removed. If the script is run after a reboot, it will generate an error when it tries to stop and uninstall cloudflared because the service does not exist and is not running. These commands now run with 2> /dev/null to suppress related error messages.
2. Not sure how to improve language around this. Any suggestions?

Thank you for your response and for updating the script to use podman!
I see that you are (now?) copying the .pem file form the ${CLOUDFLARED_DIR}/cert.pem so it is not necessary to manually place the $HOME folder as described in line 10. Maybe the comment can be deleted?

No, line 10 is just a comment saying that you should place those files in that directory. If the files are found there then the script will try to use them. If they are not found, then the script will generate a link to generate the file on Cloudflare and download it to your home directory with the expected filename. All of that happens in lines 40 - 60

Oh I see, sorry my mistake.

No apologies necessary, it means that my script could be more clear

Stop using this