Disable WAN interfaces when CARP is down

10-wancarp

#!/usr/local/bin/php

<?php

/*
    This script can be used with OPNsense when using CARP in certain circumstances where CARP is desired on the LAN side
    but where CARP cannot run on the WAN side. This script runs each time an event by CARP is generated. In the event that 
    there are multiple LAN interfaces where CARP is enabled, this script will check that all interfaces where CARP is 
    enabled are in a particular state (MASTER/BACKUP) before enabling or disabling its WAN interfaces.
*/

// this file should be saved to /usr/local/etc/rc.syshook.d/carp/10-wancarp
// some ideas taken from https://gist.github.com/taxilian/eecdc1fb17cf70e8080118cf6d8af412

/* --- EDIT AS NEEDED --- */
// my primary WAN is on the opt2 interface and secondary WAN is on the wan interface
$interfaces = array('opt2', 'wan');

require_once("config.inc");
require_once("interfaces.inc");
require_once("util.inc");

$subsystem = !empty($argv[1]) ? $argv[1] : '';
$type = !empty($argv[2]) ? $argv[2] : '';

if ($type != 'MASTER' && $type != 'BACKUP' && $type != 'INIT' && $type != 'DISABLED') {
    log_error("CARP '$type' event unknown from source '{$subsystem}'");
    exit(1);
}

if (!strstr($subsystem, '@')) {
    log_error("CARP '$type' event triggered from wrong source '{$subsystem}'");
    exit(1);
}

// Get a list of CARP statuses across interfaces that have it
$cmd = "/sbin/ifconfig -m -v | grep 'carp:' | awk '{print $2}'";
$cmd2 = "/sbin/ifconfig -m -v | grep 'carp:' | awk '{print $2}' | wc -l | tr -d ' '";
exec($cmd, $ifconfig_data, $ret);
exec($cmd2, $ifconfig_num, $ret);
$activeCARPInterfaces = $ifconfig_num[0];

// Log what the CARP subsystem is doing
// After INIT, the subsystem will become BACKUP, then MASTER if needed
if($type == 'INIT' && $activeCARPInterfaces > 0) {
  log_error("CARP on '{$subsystem}' is now INIT, but other subsystems are still active");
} else if($type == 'INIT' && $activeCARPInterfaces == 0) {
  log_error("CARP on '{$subsystem}' is now INIT, and appears to be the last active");
}
else if($type == 'MASTER' && $activeCARPInterfaces > 0) {
  log_error("CARP on '{$subsystem}' is now MASTER, but other subsystems are still active");
} else if($type == 'MASTER' && $activeCARPInterfaces == 0) {
  log_error("CARP on '{$subsystem}' is now MASTER, and appears to be the last active");
}
else if($type == 'BACKUP' && $activeCARPInterfaces > 0) {
  log_error("CARP on '{$subsystem}' is now BACKUP, other subsystems are still active");
} else if($type == 'BACKUP' && $activeCARPInterfaces == 0) {
  log_error("CARP on '{$subsystem}' is now BACKUP, and appears to be the last active");
}

// If no CARP subsystems/interfaces are active, then disable WAN interfaces
// This will generally apply when CARP is disabled and reports no statuses
if($activeCARPInterfaces == 0) {
  log_error("No CARP subsystems are active, deactivating WAN interfaces");

  foreach ($interfaces as $ifkey) {
      unset($config['interfaces'][$ifkey]['enable']);
      interface_configure(false, $ifkey, false, false);
      write_config("disable interface '$ifkey' due CARP event '$type'", false);
  }

  exit(0);
}

// Keep track of active MASTER and BACKUP instances
$masterCount = 0;
$backupCount = 0;

// Loop over $ifconfig_data and count how many are "MASTER" and how many are "BACKUP"
foreach ($ifconfig_data as $line) {
  if (strpos($line, 'MASTER') !== false) {
    $masterCount++;
  } else if (strpos($line, 'BACKUP') !== false) {
    $backupCount++;
  }
}

// Toggle WAN interfaces depending if all interfaces are MASTER or BACKUP
if ($masterCount == $activeCARPInterfaces && $masterCount > 0) {
  // The current node is all MASTER
  log_error("All CARP subsystems are MASTER, activating WAN interfaces");

  foreach ($interfaces as $ifkey) {
      $config['interfaces'][$ifkey]['enable'] = '1';
      interface_configure(false, $ifkey, false, false);
      write_config("enable interface '$ifkey' due CARP event '$type'", false);
  }
} else if ($backupCount == $activeCARPInterfaces && $backupCount > 0) {
  // The current node is all BACKUP
  log_error("Not all CARP subsystems are MASTER, deactivating WAN interfaces");

  foreach ($interfaces as $ifkey) {
      unset($config['interfaces'][$ifkey]['enable']);
      interface_configure(false, $ifkey, false, false);
      write_config("disable interface '$ifkey' due CARP event '$type'", false);
  }
}

?>

Loving the look of this but I'm a little perplexed as to the purpose of the call to interface_configure when $config['interfaces'][$ifkey] is unset. The source for interface_configure has immediate check if $config['interfaces'][$interface]['enable'] is set. It does nothing except return an empty array if the enable key is unset, which will always be true on line 68 and 104. Should this invocation be replaced with a call to interface_reset($interface) or is it omissible? I struggle to see how the interface would be disabled otherwise. I imagine the call to write_config is what is causing the changes to take effect rather than interface_configure. I'm going to try this out now with interface_reset and I'll see how it goes. Any thoughts?

Did you find a solution? Perhaps it is the same problem which i mentioned here?

@skl283 I'm having a similar issue; IPv4 works instantly but my backup router won't pick up IPv6 information even if I reload using the command or gui. I have a very limited understanding of IPv6, not really sure how Router Advertisements work/why my backup router isn't working when my primary works each time. I have my IPv6 DUID set the same on each machine and I use the same MAC on my single WAN on both machines. Is it wise to use the same DUID for IPv6 in this scenario? I have Interfaces > Settings > IPv6 DHCP > Prevent Release enabled on both machines.

I also notice when I enter persistent CARP maintenance mode, I get a warning from OPNsense that an issue was detected with the machine and it has been demoted to backup, which usually doesn't show up, the machine would normally just go into backup. I have to enter persistent CARP maintenance mode again for it to properly engage, then press the button again to leave it. Not sure what's causing that warning, though a require_once("system.inc"); helped me fix some errors I was seeing reported by OPNsense, though not all.