Skip to content

Instantly share code, notes, and snippets.

@leecardona
Last active October 24, 2017 20:42
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save leecardona/e9230cc42d3a2ab557087d8a63087450 to your computer and use it in GitHub Desktop.
Save leecardona/e9230cc42d3a2ab557087d8a63087450 to your computer and use it in GitHub Desktop.
Timo,
After some additional investigation, I was able to discover that the version of frr I was using (3.0rc1 from Aug 9th)
lacked some recent fixes to route distribution and other nhrp/bgp fixes and updates.
I have since rebuilt my set up with the latest frr build of #1913 Oct. 10th this past weekend.
Source code from: https://ci1.netdef.org/browse/FRR-FRRPULLREQ-1913/
Here is the topo:
10.19.0.1/16
+-------[eth1]-------+
| |
| Hub |
| |
| 192.168.0.1/32 |
| [gre1] |
+-------[eth0]-------+
10.10.10.5/32
|
|
|
+--------------------+-------------------------+
| |
| |
10.10.10.1/32 10.10.10.6/32
+--------[eth0]------+ +--------[eth0]------+
| [gre1] | | [gre1] |
| 192.168.1.1/32 | | 192.168.0.6/32 |
| | | |
| Spoke 1 | | Spoke 2 |
| | | |
| | | |
+--------[eth1]------+ +--------[eth1]------+
10.50.0.1/16 172.31.0.6/16
| |
| |
=============== ===============
subnet: 10.50.0.0/16 subnet: 172.31.0.0/16
Everything looks stable and debug log comes up clean.
Now when pinging from tunnel interfaces, spoke to spoke (S2S) tunnel comes up and switches from phase 1 (via hub) to
phase 3 (spoke to spoke) after ~ 10 pings following hub redirect.
However, I can not get phase 3 spoke to spoke to initiate if the pings are from the back-end networks (10.50/16 or 172.31/16).
Specifically, starting from an initial state where both spokes have registered with the hub and hub<->spoke tunnels are up.
ibgp is working with both spokes getting all routes from hub - e.g. tunnels and back-end networks routes.
If I then do a ping from either spoke's tunnel (gre1) interface to the other spoke's tunnel (gre1) interface, then as
expected:
- redirect happens - iptables shows packets for the NFLOG rule in forward table
- IPSec SA establish between the spokes
- direct spoke next-hop route [N] route gets installed
- traffic stops going over the hub and goes direct over the new spoke-to-spoke (S2S) tunnel
If I then - after this e.g. S2S Tunnels is up already - ping from either spoke's tunnel (gre1) interface or either spoke's
back-end network interface (eth1) to the other spokes back-end interface (eth1), then:
- (I think everything) works as it should as traffic uses the existing tunnel and not the hub and appears to works fine.
If I however, start from the initial state (with no exisitng S2S tunnel up) and do a ping from either spoke's tunnel (gre1)
interface or either spoke's back-end network interface (eth1) to the other spokes back-end interface (eth1), then:
- Not sure if redirect happens correctly - I can see the shortcut for the target back-end network gets installed in the
originating spoke via 'sh ip nhrp shorcut'
- An 'Invalid' entry is added to the originating spoke's cache ('sh ip nhrp cache') with a missing nbma IP for the remote
spoke.
- IPSec SA does not establish at all
- traffic flows and reaches the other spoke but over the hub only
In summary,
- it appears phase 1 works fine whether its tunnel to tunnel, back-end to back-end, tunnel to back-end or back-end to
tunnel over the hub
- it appears phase 3 S2S works only when its tunnel to tunnel
- phase 3 S2S does not work if its back-end to back-end, tunnel to back-end or back-end to tunnel
I'm just not sure if it's a configuration issue on my end with nhrp or bgp or if this is a bug?
Here is a link to a gist that has the configs of the hub and spokes plus debug output.
https://gist.github.com/leecardona/e9230cc42d3a2ab557087d8a63087450
============================
Hub Configs and CLI Output
============================
========================
/etc/network/interfaces
# NETWORK INTERFACES CONFIG USED AT SYSTEM START
# LOOPBACK INTERFACE CONFIG
auto lo
iface lo inet loopback
# ETH0 INTERFACE CONFIG
auto eth0
iface eth0 inet static
address 10.10.10.5/32
dns-nameservers 8.8.8.8 8.8.4.4
post-up ip route add default dev eth0
post-down ip route delete default dev eth0
# ETH1 INTERFACE CONFIG
auto eth1
iface eth1 inet static
address 10.19.0.1/16
auto gre1
iface gre1 inet static
pre-up ip tunnel add $IFACE mode gre key 42 ttl 64 dev eth0 || true
address 192.168.0.1
netmask 255.255.255.255
post-down ip tunnel del $IFACE || true
========================
/etc/iptables.rules
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -i lo -j ACCEPT
-A INPUT -p esp -j ACCEPT
-A INPUT -p udp -m udp --sport 500 --dport 500 -j ACCEPT
-A INPUT -p udp -m udp --sport 4500 --dport 4500 -j ACCEPT
-A INPUT -s 10.10.10.0/24 -i eth0 -j ACCEPT
-A INPUT -s 192.168.0.0/16 -i gre1 -j ACCEPT
-A INPUT -s 172.31.0.0/16 -i gre1 -j ACCEPT
-A INPUT -s 10.50.0.0/16 -i gre1 -j ACCEPT
-A INPUT -j DROP
-A FORWARD -i gre1 -o gre1 -m hashlimit --hashlimit-upto 4/min --hashlimit-burst 1 --hashlimit-mode srcip,dstip --hashlimit-srcmask 16 --hashlimit-dstmask 16 --hashlimit-name loglimit-0 -j NFLOG --nflog-group 1 --nflog-range 128
-A FORWARD -s 192.168.0.0/16 -j ACCEPT
-A FORWARD -s 172.31.0.0/16 -j ACCEPT
-A FORWARD -s 10.50.0.0/16 -j ACCEPT
COMMIT
========================
/etc/swanctl/swanctl.conf
connections {
dmvpn {
version = 2
pull = no
mobike = no
dpd_delay = 15
dpd_timeout = 30
fragmentation = yes
unique = replace
rekey_time = 4h
reauth_time = 13h
proposals = aes256-sha512-ecp384
local {
auth = psk
id = 10.10.10.5
}
remote {
auth = psk
}
children {
dmvpn {
esp_proposals = aes256-sha512-ecp384
local_ts = dynamic[gre]
remote_ts = dynamic[gre]
inactivity = 90m
rekey_time = 100m
mode = transport
dpd_action = clear
}
}
}
}
========================
/etc/frr/frr.conf
frr version 3.0-dev-FRR-Version-buildtest-1913
frr defaults traditional
hostname Hub
no log monitor
no ipv6 forwarding
nhrp nflog-group 1
username cumulus nopassword
!
service integrated-vtysh-config
!
debug nhrp all
!
log file /var/log/frr/frr.log
!
log syslog informational
!
interface eth0
description OUTSIDE INTERFACE
ip address 10.10.10.5/32
!
interface eth1
description HUB BACKEND
ip address 10.19.0.1/16
!
interface gre1
ip address 192.168.0.1/32
ip nhrp network-id 1
ip nhrp nhs dynamic nbma 10.10.10.5
ip nhrp redirect
ip nhrp registration no-unique
no link-detect
tunnel protection vici profile dmvpn
tunnel source eth0
!
router bgp 65000
bgp router-id 192.168.0.1
no bgp default ipv4-unicast
bgp default show-hostname
neighbor DMVPN peer-group
neighbor DMVPN remote-as 65000
neighbor DMVPN disable-connected-check
neighbor DMVPN advertisement-interval 1
bgp listen range 192.168.0.0/16 peer-group DMVPN
!
address-family ipv4 unicast
redistribute nhrp
neighbor DMVPN activate
neighbor DMVPN route-reflector-client
neighbor DMVPN soft-reconfiguration inbound
exit-address-family
vnc defaults
response-lifetime 3600
exit-vnc
!
line vty
!
========================
vtysh - sh ip bgp
Hub# sh ip bgp
BGP table version is 29, local router ID is 192.168.0.1
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*>i10.50.0.0/16 192.168.1.1 0 100 0 i
*>i172.31.0.0 192.168.0.6 0 100 0 i
*> 192.168.0.6/32 0.0.0.0 0 32768 ?
*> 192.168.1.1/32 0.0.0.0 0 32768 ?
Displayed 5 routes and 5 total paths
========================
vtysh - sh ip route
Hub# sh ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, P - PIM, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel,
> - selected route, * - FIB route
K>* 0.0.0.0/0 [0/0] is directly connected, eth0, 05:06:38
B> 10.50.0.0/16 [200/0] via 192.168.1.1 (recursive), 04:54:22
* via 192.168.1.1, gre1 onlink, 04:54:22
C>* 10.10.10.5/32 is directly connected, eth0, 05:06:38
B> 172.31.0.0/16 [200/0] via 192.168.0.6 (recursive), 04:54:27
* via 192.168.0.6, gre1 onlink, 04:54:27
C>* 192.168.0.1/32 is directly connected, gre1, 05:06:38
N>* 192.168.0.6/32 [10/0] is directly connected, gre1, 04:54:27
N>* 192.168.1.1/32 [10/0] is directly connected, gre1, 04:54:23
C>* 10.19.0.0/16 is directly connected, eth1, 05:06:38
========================
vtysh - sh dmvpn
Hub# sh dmvpn
Src Dst Flags SAs Identity
10.10.10.5 10.10.10.6 n 1 10.10.10.6
10.10.10.5 10.10.10.1 n 1 10.10.10.1
========================
vtysh - sh ip nhrp shortcut
Hub# sh ip nhrp shortcut
% No entries
========================
vtysh - sh ip nhrp cache
Hub# sh ip nhrp cache
Iface Type Protocol NBMA Flags Identity
gre1 dynamic 192.168.1.1 10.10.10.1 UT 10.10.10.1
gre1 dynamic 192.168.0.6 10.10.10.6 UT 10.10.10.6
gre1 local 192.168.0.1 - -
========================
swanctl -l
Hub:/# swanctl -l
dmvpn: #10, ESTABLISHED, IKEv2, 2c141c5a6bc80b14_i 84e491e6b7fd4051_r*
local '10.10.10.5' @ 10.10.10.5[500]
remote '10.10.10.1' @ 10.10.10.1[500]
AES_CBC-256/HMAC_SHA2_512_256/PRF_HMAC_SHA2_512/ECP_384
established 4794s ago, rekeying in 6346s, reauth in 27161s
dmvpn: #13, reqid 8, INSTALLED, TRANSPORT, ESP:AES_CBC-256/HMAC_SHA2_512_256/ECP_384
installed 1432s ago, rekeying in 4243s, expires in 5168s
in cbaf72f1, 3316 bytes, 47 packets, 48s ago
out c50060f3, 2676 bytes, 36 packets, 48s ago
local 10.10.10.5/32[gre]
remote 10.10.10.1/32[gre]
dmvpn: #9, ESTABLISHED, IKEv2, d00ac4e1ee372fb9_i 3d7cd299325acf35_r*
local '10.10.10.5' @ 10.10.10.5[500]
remote '10.10.10.6' @ 10.10.10.6[500]
AES_CBC-256/HMAC_SHA2_512_256/PRF_HMAC_SHA2_512/ECP_384
established 6755s ago, rekeying in 4083s, reauth in 25572s
dmvpn: #14, reqid 7, INSTALLED, TRANSPORT, ESP:AES_CBC-256/HMAC_SHA2_512_256/ECP_384
installed 1379s ago, rekeying in 4184s, expires in 5221s
in c075a2e7, 3237 bytes, 46 packets, 53s ago
out cab03e52, 2597 bytes, 35 packets, 53s ago
local 10.10.10.5/32[gre]
remote 10.10.10.6/32[gre]
============================
Spoke 1 Configs and CLI Output
============================
========================
/etc/network/interfaces
# NETWORK INTERFACES CONFIG USED AT SYSTEM START
# LOOPBACK INTERFACE CONFIG
auto lo
iface lo inet loopback
# ETH0 INTERFACE CONFIG
auto eth0
iface eth0 inet static
address 10.10.10.1/32
dns-nameservers 8.8.8.8 8.8.4.4
post-up ip route add default dev eth0
post-down ip route delete default dev eth0
# ETH1 INTERFACE CONFIG
auto eth1
iface eth1 inet static
address 10.50.0.1/16
auto gre1
iface gre1 inet static
pre-up ip tunnel add $IFACE mode gre key 42 ttl 64 dev eth0 || true
address 192.168.1.1
netmask 255.255.255.255
post-down ip tunnel del $IFACE || true
========================
/etc/swanctl/swanctl.conf
connections {
dmvpn {
version = 2
pull = no
mobike = no
dpd_delay = 15
dpd_timeout = 30
fragmentation = yes
unique = replace
rekey_time = 4h
reauth_time = 13h
proposals = aes256-sha512-ecp384
local {
auth = psk
id = 10.10.10.1
}
remote {
auth = psk
}
children {
dmvpn {
esp_proposals = aes256-sha512-ecp384
local_ts = dynamic[gre]
remote_ts = dynamic[gre]
inactivity = 90m
rekey_time = 100m
mode = transport
dpd_action = clear
}
}
}
}
========================
/etc/frr/frr.conf
frr version 3.1-dev-FRR-Version-buildtest-1913
frr defaults traditional
hostname Spoke1
no log monitor
no ipv6 forwarding
username cumulus nopassword
!
service integrated-vtysh-config
!
debug nhrp all
!
log file /var/log/frr/frr.log
!
log syslog informational
!
interface eth0
description OUTSIDE INTERFACE
ip address 10.10.10.1/32
!
interface eth1
description BACKEND INTERFACE
ip address 10.50.0.1/16
!
interface gre1
ip address 192.168.1.1/32
ip nhrp network-id 1
ip nhrp nhs dynamic nbma 10.10.10.5
ip nhrp registration no-unique
ip nhrp shortcut
no link-detect
tunnel protection vici profile dmvpn
tunnel source eth0
!
router bgp 65000
bgp router-id 192.168.1.1
no bgp default ipv4-unicast
bgp default show-hostname
neighbor DMVPN peer-group
neighbor DMVPN remote-as 65000
neighbor DMVPN disable-connected-check
neighbor DMVPN advertisement-interval 1
neighbor 192.168.0.1 peer-group DMVPN
!
address-family ipv4 unicast
network 10.50.0.0/16
neighbor DMVPN activate
neighbor DMVPN next-hop-self
neighbor DMVPN soft-reconfiguration inbound
exit-address-family
vnc defaults
response-lifetime 3600
exit-vnc
!
line vty
!
========================
vtysh - sh ip bgp
Spoke1# sh ip bgp
BGP table version is 5, local router ID is 192.168.1.1
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*> 10.50.0.0/16 0.0.0.0 0 32768 i
*>i172.31.0.0 192.168.0.1 0 100 0 i
*>i192.168.0.6/32 192.168.0.1 0 100 0 ?
*>i192.168.1.1/32 192.168.0.1 0 100 0 ?
Displayed 5 routes and 5 total paths
========================
vtysh - sh ip route
Spoke1# sh ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, P - PIM, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel,
> - selected route, * - FIB route
K>* 0.0.0.0/0 [0/0] is directly connected, eth0, 05:13:08
C>* 10.50.0.0/16 is directly connected, eth1, 05:13:08
C>* 10.10.10.1/32 is directly connected, eth0, 05:13:08
B> 172.31.0.0/16 [200/0] via 192.168.0.6 (recursive), 00:02:53
* via 192.168.0.1, gre1 onlink, 00:02:53
N>* 192.168.0.1/32 [10/0] is directly connected, gre1, 05:13:06
B> 192.168.0.6/32 [200/0] via 192.168.0.1 (recursive), 05:13:05
* via 192.168.0.1, gre1 onlink, 05:13:05
B 192.168.1.1/32 [200/0] via 192.168.0.1, 05:13:05
C>* 192.168.1.1/32 is directly connected, gre1, 05:13:08
========================
vtysh - sh dmvpn
Spoke1# sh dmvpn
Src Dst Flags SAs Identity
10.10.10.1 10.10.10.5 n 1 10.10.10.5
========================
vtysh - sh ip nhrp shortcut
Spoke1# sh ip nhrp shortcut
% No entries
========================
vtysh - sh ip nhrp cache
Spoke1# sh ip nhrp cache
Iface Type Protocol NBMA Flags Identity
gre1 local 192.168.1.1 - -
gre1 nhs 192.168.0.1 10.10.10.5 UT 10.10.10.5
========================
swanctl -l
Spoke1:/# swanctl -l
dmvpn: #1, ESTABLISHED, IKEv2, 04622f9045243ade_i* 32710853f928ddb8_r
local '10.10.10.1' @ 10.10.10.1[500]
remote '10.10.10.5' @ 10.10.10.5[500]
AES_CBC-256/HMAC_SHA2_512_256/PRF_HMAC_SHA2_512/ECP_384
established 20s ago, rekeying in 11083s, reauth in 41277s
dmvpn: #1, reqid 1, INSTALLED, TRANSPORT, ESP:AES_CBC-256/HMAC_SHA2_512_256
installed 20s ago, rekeying in 5747s, expires in 6580s
in c8a58327, 1408 bytes, 14 packets, 5s ago
out c49d6740, 1126 bytes, 15 packets, 5s ago
local 10.10.10.1/32[gre]
remote 10.10.10.5/32[gre]
============================
Spoke 2 Configs and CLI Output
============================
========================
/etc/network/interfaces
# NETWORK INTERFACES CONFIG USED AT SYSTEM START
# LOOPBACK INTERFACE CONFIG
auto lo
iface lo inet loopback
# ETH0 INTERFACE CONFIG
auto eth0
iface eth0 inet static
address 10.10.10.6/32
dns-nameservers 8.8.8.8 8.8.4.4
post-up ip route add default dev eth0
post-down ip route delete default dev eth0
# ETH1 INTERFACE CONFIG
auto eth1
iface eth1 inet static
address 172.31.0.6/16
auto gre1
iface gre1 inet static
pre-up ip tunnel add $IFACE mode gre key 42 ttl 64 dev eth0 || true
address 192.168.0.6
netmask 255.255.255.255
post-down ip tunnel del $IFACE || true
========================
/etc/swanctl/swanctl.conf
Spoke2:/# cat /etc/swanctl/swanctl.conf
connections {
dmvpn {
version = 2
pull = no
mobike = no
dpd_delay = 15
dpd_timeout = 30
fragmentation = yes
unique = replace
rekey_time = 4h
reauth_time = 13h
proposals = aes256-sha512-ecp384
local {
auth = psk
id = 10.10.10.6
}
remote {
auth = psk
}
children {
dmvpn {
esp_proposals = aes256-sha512-ecp384
local_ts = dynamic[gre]
remote_ts = dynamic[gre]
inactivity = 90m
rekey_time = 100m
mode = transport
dpd_action = clear
}
}
}
}
========================
/etc/frr/frr.conf
frr version 3.1-dev-FRR-Version-buildtest-1913
frr defaults traditional
hostname Spoke2
no log monitor
no ipv6 forwarding
username cumulus nopassword
!
service integrated-vtysh-config
!
debug nhrp all
!
log file /var/log/frr/frr.log
!
log syslog informational
!
interface eth0
description OUTSIDE INTERFACE
ip address 10.10.10.6/32
!
interface eth1
description Inside interface
ip address 172.31.0.6/16
!
interface gre1
ip address 192.168.0.6/32
ip nhrp network-id 1
ip nhrp nhs dynamic nbma 10.10.10.5
ip nhrp registration no-unique
ip nhrp shortcut
no link-detect
tunnel protection vici profile dmvpn
tunnel source eth0
!
router bgp 65000
bgp router-id 192.168.0.6
no bgp default ipv4-unicast
bgp default show-hostname
neighbor DMVPN peer-group
neighbor DMVPN remote-as 65000
neighbor DMVPN disable-connected-check
neighbor DMVPN advertisement-interval 1
neighbor 192.168.0.1 peer-group DMVPN
!
address-family ipv4 unicast
network 172.31.0.0/16
neighbor DMVPN activate
neighbor DMVPN next-hop-self
neighbor DMVPN soft-reconfiguration inbound
exit-address-family
vnc defaults
response-lifetime 3600
exit-vnc
!
line vty
!
========================
vtysh - sh ip bgp
Spoke2# sh ip bgp
BGP table version is 4, local router ID is 192.168.0.6
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop Metric LocPrf Weight Path
*>i10.50.0.0/16 192.168.1.1 0 100 0 i
*> 172.31.0.0 0.0.0.0 0 32768 i
*>i192.168.0.6/32 192.168.0.1 0 100 0 ?
*>i192.168.1.1/32 192.168.0.1 0 100 0 ?
Displayed 4 routes and 4 total paths
========================
vtysh - sh ip route
Spoke2# sh ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, P - PIM, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel,
> - selected route, * - FIB route
K>* 0.0.0.0/0 [0/0] is directly connected, eth0, 00:06:05
B> 10.50.0.0/16 [200/0] via 192.168.1.1 (recursive), 00:06:03
* via 192.168.0.1, gre1 onlink, 00:06:03
C>* 10.10.10.6/32 is directly connected, eth0, 00:06:05
C>* 172.31.0.0/16 is directly connected, eth1, 00:06:05
N>* 192.168.0.1/32 [10/0] is directly connected, gre1, 00:06:03
B 192.168.0.6/32 [200/0] via 192.168.0.1, 00:06:03
C>* 192.168.0.6/32 is directly connected, gre1, 00:06:05
B> 192.168.1.1/32 [200/0] via 192.168.0.1 (recursive), 00:06:03
* via 192.168.0.1, gre1 onlink, 00:06:03
========================
vtysh - sh dmvpn
Spoke2# sh dmvpn
Src Dst Flags SAs Identity
10.10.10.6 10.10.10.5 n 1 10.10.10.5
========================
vtysh - sh ip nhrp shortcut
Spoke2# sh ip nhrp shortcut
% No entries
========================
vtysh - sh ip nhrp cache
Spoke2# sh ip nhrp cache
Iface Type Protocol NBMA Flags Identity
gre1 local 192.168.0.6 - -
gre1 nhs 192.168.0.1 10.10.10.5 UT 10.10.10.5
========================
swanctl -l
Spoke2:/# swanctl -l
dmvpn: #1, ESTABLISHED, IKEv2, 2fc87ee6dcf4703c_i* 91c9015b2d96d5ad_r
local '10.10.10.6' @ 10.10.10.6[500]
remote '10.10.10.5' @ 10.10.10.5[500]
AES_CBC-256/HMAC_SHA2_512_256/PRF_HMAC_SHA2_512/ECP_384
established 451s ago, rekeying in 9271s, reauth in 40439s
dmvpn: #1, reqid 1, INSTALLED, TRANSPORT, ESP:AES_CBC-256/HMAC_SHA2_512_256
installed 451s ago, rekeying in 5111s, expires in 6149s
in c6aaed8a, 1781 bytes, 21 packets, 31s ago
out c439289a, 1979 bytes, 27 packets, 31s ago
local 10.10.10.6/32[gre]
remote 10.10.10.5/32[gre]
============================
Debug Output from start to
Registration completed
============================
=========
Hub
=========
Hub:/# cat /var/log/frr/frr.log
2017/10/24 19:30:12 NHRP: gre1: IPv4 address changed to 192.168.0.1/32
2017/10/24 19:30:12 NHRP: cache: gre1 192.168.0.1: accept
2017/10/24 19:30:12 NHRP: netlink-link-gre: get-info 4
2017/10/24 19:30:12 NHRP: netlink-link-gre: ifindex 4, receive msg_type 16, msg_flags 0
2017/10/24 19:30:12 NHRP: gre1: GRE: 2a000000 38 0
2017/10/24 19:30:13 NHRP: [0x1a217c0] Resolving '10.10.10.5'
2017/10/24 19:30:13 NHRP: [0x1a217c0] Resolved with 1 results
2017/10/24 19:30:26 NHRP: PACKET: Recv 10.10.10.1 -> 10.10.10.5
2017/10/24 19:30:26 NHRP: Recv Registration-Request(3) 192.168.1.1 -> 192.168.1.1
2017/10/24 19:30:26 NHRP: !LOCAL Registration-Request(3) 192.168.1.1 -> 192.168.1.1
2017/10/24 19:30:26 NHRP: Parsing and replying to Registration Req
2017/10/24 19:30:26 NHRP: cache: gre1 192.168.1.1: accept
2017/10/24 19:30:26 NHRP: Zebra send: route add 192.168.1.1/32 nexthop <onlink> metric 0 count 1 dev gre1
2017/10/24 19:30:26 NHRP: Send Registration-Reply(4) 192.168.0.1 -> 192.168.1.1
2017/10/24 19:30:26 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.1
2017/10/24 19:30:26 NHRP: Netlink: Received msg_type 28, msg_flags 0
2017/10/24 19:30:26 NHRP: Netlink: update 192.168.1.1 dev gre1 nud 2
2017/10/24 19:30:26 NHRP: if-route-add: 192.168.1.1/32 via 0.0.0.0 dev gre1
2017/10/24 19:30:27 NHRP: if-route-add: 10.50.0.0/16 via 192.168.1.1 dev (none)
2017/10/24 19:30:36 NHRP: cache: gre1 192.168.1.1: timeout
2017/10/24 19:30:46 NHRP: PACKET: Recv 10.10.10.6 -> 10.10.10.5
2017/10/24 19:30:46 NHRP: Recv Registration-Request(3) 192.168.0.6 -> 192.168.0.6
2017/10/24 19:30:46 NHRP: !LOCAL Registration-Request(3) 192.168.0.6 -> 192.168.0.6
2017/10/24 19:30:46 NHRP: Parsing and replying to Registration Req
2017/10/24 19:30:46 NHRP: cache: gre1 192.168.0.6: accept
2017/10/24 19:30:46 NHRP: Zebra send: route add 192.168.0.6/32 nexthop <onlink> metric 0 count 1 dev gre1
2017/10/24 19:30:46 NHRP: Send Registration-Reply(4) 192.168.0.1 -> 192.168.0.6
2017/10/24 19:30:46 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.6
2017/10/24 19:30:46 NHRP: if-route-add: 192.168.0.6/32 via 0.0.0.0 dev gre1
2017/10/24 19:30:46 NHRP: if-route-add: 172.31.0.0/16 via 192.168.0.6 dev (none)
2017/10/24 19:30:56 NHRP: cache: gre1 192.168.0.6: timeout
=========
Spoke 1
=========
Spoke1:/# cat /var/log/frr/frr.log
2017/10/24 19:30:25 NHRP: gre1: IPv4 address changed to 192.168.1.1/32
2017/10/24 19:30:25 NHRP: cache: gre1 192.168.1.1: accept
2017/10/24 19:30:25 NHRP: netlink-link-gre: get-info 4
2017/10/24 19:30:25 NHRP: netlink-link-gre: ifindex 4, receive msg_type 16, msg_flags 0
2017/10/24 19:30:25 NHRP: gre1: GRE: 2a000000 3c 0
2017/10/24 19:30:26 NHRP: [0x16f0550] Resolving '10.10.10.5'
2017/10/24 19:30:26 NHRP: [0x16f0550] Resolved with 1 results
2017/10/24 19:30:26 NHRP: NHS: Register 192.168.1.1 -> 192.168.1.1 (timeout 1)
2017/10/24 19:30:26 NHRP: Send Registration-Request(3) 192.168.1.1 -> 192.168.1.1
2017/10/24 19:30:26 NHRP: PACKET: Send 10.10.10.1 -> 10.10.10.5
2017/10/24 19:30:26 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.1
2017/10/24 19:30:26 NHRP: Recv Registration-Reply(4) 192.168.0.1 -> 192.168.1.1
2017/10/24 19:30:26 NHRP: !LOCAL Registration-Reply(4) 192.168.0.1 -> 192.168.1.1
2017/10/24 19:30:26 NHRP: NHS: Reg.reply received
2017/10/24 19:30:26 NHRP: NHS: CIE registration: 192.168.1.1: 0
2017/10/24 19:30:26 NHRP: cache: gre1 192.168.0.1: accept
2017/10/24 19:30:26 NHRP: Zebra send: route add 192.168.0.1/32 nexthop <onlink> metric 0 count 1 dev gre1
2017/10/24 19:30:26 NHRP: Netlink: Received msg_type 28, msg_flags 0
2017/10/24 19:30:26 NHRP: Netlink: update 192.168.0.1 dev gre1 nud 2
2017/10/24 19:30:26 NHRP: if-route-add: 192.168.0.1/32 via 0.0.0.0 dev gre1
2017/10/24 19:30:36 NHRP: cache: gre1 192.168.0.1: timeout
2017/10/24 19:30:46 NHRP: if-route-add: 192.168.0.6/32 via 192.168.0.1 dev (none)
2017/10/24 19:30:47 NHRP: if-route-add: 172.31.0.0/16 via 192.168.0.6 dev (none)
=========
Spoke 2
=========
Spoke2:/# cat /var/log/frr/frr.log
2017/10/24 19:30:45 NHRP: gre1: IPv4 address changed to 192.168.0.6/32
2017/10/24 19:30:45 NHRP: cache: gre1 192.168.0.6: accept
2017/10/24 19:30:45 NHRP: netlink-link-gre: get-info 4
2017/10/24 19:30:45 NHRP: netlink-link-gre: ifindex 4, receive msg_type 16, msg_flags 0
2017/10/24 19:30:45 NHRP: gre1: GRE: 2a000000 40 0
2017/10/24 19:30:46 NHRP: [0xe85610] Resolving '10.10.10.5'
2017/10/24 19:30:46 NHRP: [0xe85610] Resolved with 1 results
2017/10/24 19:30:46 NHRP: NHS: Register 192.168.0.6 -> 192.168.0.6 (timeout 1)
2017/10/24 19:30:46 NHRP: Send Registration-Request(3) 192.168.0.6 -> 192.168.0.6
2017/10/24 19:30:46 NHRP: PACKET: Send 10.10.10.6 -> 10.10.10.5
2017/10/24 19:30:46 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.6
2017/10/24 19:30:46 NHRP: Recv Registration-Reply(4) 192.168.0.1 -> 192.168.0.6
2017/10/24 19:30:46 NHRP: !LOCAL Registration-Reply(4) 192.168.0.1 -> 192.168.0.6
2017/10/24 19:30:46 NHRP: NHS: Reg.reply received
2017/10/24 19:30:46 NHRP: NHS: CIE registration: 192.168.0.6: 0
2017/10/24 19:30:46 NHRP: cache: gre1 192.168.0.1: accept
2017/10/24 19:30:46 NHRP: Zebra send: route add 192.168.0.1/32 nexthop <onlink> metric 0 count 1 dev gre1
2017/10/24 19:30:46 NHRP: Netlink: Received msg_type 28, msg_flags 0
2017/10/24 19:30:46 NHRP: Netlink: update 192.168.0.1 dev gre1 nud 2
2017/10/24 19:30:46 NHRP: if-route-add: 192.168.0.1/32 via 0.0.0.0 dev gre1
2017/10/24 19:30:46 NHRP: if-route-add: 192.168.1.1/32 via 192.168.0.1 dev (none)
2017/10/24 19:30:46 NHRP: if-route-add: 10.50.0.0/16 via 192.168.1.1 dev (none)
2017/10/24 19:30:56 NHRP: cache: gre1 192.168.0.1: timeout
============================
Debug Output from
Registration completed to
net to net ping - DOES NOT WORK AS EXPECTED e.g. WONT DO PHASE 3
============================
=========
Hub
=========
2017/10/24 19:38:16 NHRP: Netlink-log: Received msg_type 1024, msg_flags 0
2017/10/24 19:38:16 NHRP: lookup 10.50.0.1/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.1.1/32: nhrp_if=gre1
2017/10/24 19:38:16 NHRP: Send Traffic Indication to 10.10.10.1 (online=1) about packet to 10.50.0.1
2017/10/24 19:38:16 NHRP: Send Traffic-Indication(8) 192.168.0.1 -> 10.50.0.1
2017/10/24 19:38:16 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.1
2017/10/24 19:38:16 NHRP: Netlink-log: Received msg_type 1024, msg_flags 0
2017/10/24 19:38:16 NHRP: lookup 172.31.0.6/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.0.6/32: nhrp_if=gre1
2017/10/24 19:38:16 NHRP: Send Traffic Indication to 10.10.10.6 (online=1) about packet to 172.31.0.6
2017/10/24 19:38:16 NHRP: Send Traffic-Indication(8) 192.168.0.1 -> 172.31.0.6
2017/10/24 19:38:16 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.6
2017/10/24 19:38:16 NHRP: Netlink-log: Received msg_type 3, msg_flags 0
2017/10/24 19:38:16 NHRP: PACKET: Recv 10.10.10.1 -> 10.10.10.5
2017/10/24 19:38:16 NHRP: Recv Resolution-Request(1) 192.168.1.1 -> 172.31.0.6
2017/10/24 19:38:16 NHRP: lookup 172.31.0.6/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.0.6/32: nhrp_if=gre1
2017/10/24 19:38:16 NHRP: Send Resolution-Request(1) 192.168.1.1 -> 172.31.0.6
2017/10/24 19:38:16 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.6
2017/10/24 19:38:16 NHRP: PACKET: Recv 10.10.10.6 -> 10.10.10.5
2017/10/24 19:38:16 NHRP: Recv Resolution-Request(1) 192.168.0.6 -> 10.50.0.1
2017/10/24 19:38:16 NHRP: lookup 10.50.0.1/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.1.1/32: nhrp_if=gre1
2017/10/24 19:38:16 NHRP: Send Resolution-Request(1) 192.168.0.6 -> 10.50.0.1
2017/10/24 19:38:16 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.1
2017/10/24 19:38:16 NHRP: PACKET: Recv 10.10.10.6 -> 10.10.10.5
2017/10/24 19:38:16 NHRP: Recv Resolution-Reply(2) 172.31.0.6 -> 192.168.1.1
2017/10/24 19:38:16 NHRP: lookup 192.168.1.1/32: nhrp_if=gre1
2017/10/24 19:38:16 NHRP: Send Resolution-Reply(2) 172.31.0.6 -> 192.168.1.1
2017/10/24 19:38:16 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.1
2017/10/24 19:38:16 NHRP: PACKET: Recv 10.10.10.1 -> 10.10.10.5
2017/10/24 19:38:16 NHRP: Recv Resolution-Reply(2) 10.50.0.1 -> 192.168.0.6
2017/10/24 19:38:16 NHRP: lookup 192.168.0.6/32: nhrp_if=gre1
2017/10/24 19:38:16 NHRP: Send Resolution-Reply(2) 10.50.0.1 -> 192.168.0.6
2017/10/24 19:38:16 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.6
=========
Spoke 1
=========
2017/10/24 19:38:16 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.1
2017/10/24 19:38:16 NHRP: Recv Traffic-Indication(8) 192.168.0.1 -> 10.50.0.1
2017/10/24 19:38:16 NHRP: lookup 10.50.0.1/32: zebra route dev eth1
2017/10/24 19:38:16 NHRP: Traffic Indication from 192.168.0.1 about packet to 172.31.0.6: trying shortcut
2017/10/24 19:38:16 NHRP: Shortcut 172.31.0.6/32 created
2017/10/24 19:38:16 NHRP: lookup 172.31.0.6/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.0.6/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.0.1/32: nhrp_if=gre1
2017/10/24 19:38:16 NHRP: Send Resolution-Request(1) 192.168.1.1 -> 172.31.0.6
2017/10/24 19:38:16 NHRP: PACKET: Send 10.10.10.1 -> 10.10.10.5
2017/10/24 19:38:16 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.1
2017/10/24 19:38:16 NHRP: Recv Resolution-Request(1) 192.168.0.6 -> 10.50.0.1
2017/10/24 19:38:16 NHRP: lookup 10.50.0.1/32: zebra route dev eth1
2017/10/24 19:38:16 NHRP: Parsing and replying to Resolution Req
2017/10/24 19:38:16 NHRP: lookup 192.168.0.6/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.0.1/32: nhrp_if=gre1
2017/10/24 19:38:16 NHRP: Send Resolution-Reply(2) 10.50.0.1 -> 192.168.0.6
2017/10/24 19:38:16 NHRP: PACKET: Send 10.10.10.1 -> 10.10.10.5
2017/10/24 19:38:16 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.1
2017/10/24 19:38:16 NHRP: Recv Resolution-Reply(2) 172.31.0.6 -> 192.168.1.1
2017/10/24 19:38:16 NHRP: !LOCAL Resolution-Reply(2) 172.31.0.6 -> 192.168.1.1
2017/10/24 19:38:16 NHRP: lookup 172.31.0.6/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.0.6/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.0.1/32: nhrp_if=gre1
2017/10/24 19:38:16 NHRP: Shortcut: 172.31.0.6/16 is at proto 192.168.0.6 cie-nbma 10.10.10.6 nat-nbma (unspec) cie-holdtime 7200
2017/10/24 19:38:16 NHRP: Shortcut 172.31.0.0/16 created
2017/10/24 19:38:16 NHRP: Shortcut: Resolution reply handled
2017/10/24 19:38:16 NHRP: VICI: Message 1, 15 bytes
2017/10/24 19:38:16 NHRP: VICI: Key 'success'='yes'
2017/10/24 19:38:16 NHRP: VICI: Message 7, 668 bytes
2017/10/24 19:38:16 NHRP: VICI: Event 'child-state-installed'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='3'
2017/10/24 19:38:16 NHRP: VICI: Key 'version'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='ESTABLISHED'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-host'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-id'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-host'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-id'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'initiator-spi'='cfec233bea08461c'
2017/10/24 19:38:16 NHRP: VICI: Key 'responder-spi'='4e59549625b7d14e'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-alg'='AES_CBC'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-keysize'='256'
2017/10/24 19:38:16 NHRP: VICI: Key 'integ-alg'='HMAC_SHA2_512_256'
2017/10/24 19:38:16 NHRP: VICI: Key 'prf-alg'='PRF_HMAC_SHA2_512'
2017/10/24 19:38:16 NHRP: VICI: Key 'dh-group'='ECP_384'
2017/10/24 19:38:16 NHRP: VICI: Key 'established'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'rekey-time'='12345'
2017/10/24 19:38:16 NHRP: VICI: Key 'reauth-time'='43016'
2017/10/24 19:38:16 NHRP: VICI: List start 'tasks-passive'
2017/10/24 19:38:16 NHRP: VICI: List item: 'CHILD_CREATE'
2017/10/24 19:38:16 NHRP: VICI: List item: 'IKE_AUTH_LIFETIME'
2017/10/24 19:38:16 NHRP: VICI: List item: 'IKE_MOBIKE'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: Section start 'child-sas'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'name'='dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='3'
2017/10/24 19:38:16 NHRP: VICI: Key 'reqid'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='INSTALLING'
2017/10/24 19:38:16 NHRP: VICI: Key 'mode'='TRANSPORT'
2017/10/24 19:38:16 NHRP: VICI: List start 'local-ts'
2017/10/24 19:38:16 NHRP: VICI: List item: '10.10.10.1/32[gre]'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: List start 'remote-ts'
2017/10/24 19:38:16 NHRP: VICI: List item: '10.10.10.6/32[gre]'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Message 7, 670 bytes
2017/10/24 19:38:16 NHRP: VICI: Event 'child-state-installed'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'version'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='ESTABLISHED'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-host'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-id'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-host'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-id'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'initiator'='yes'
2017/10/24 19:38:16 NHRP: VICI: Key 'initiator-spi'='10576792c380167f'
2017/10/24 19:38:16 NHRP: VICI: Key 'responder-spi'='59b1b0ab0d68784e'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-alg'='AES_CBC'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-keysize'='256'
2017/10/24 19:38:16 NHRP: VICI: Key 'integ-alg'='HMAC_SHA2_512_256'
2017/10/24 19:38:16 NHRP: VICI: Key 'prf-alg'='PRF_HMAC_SHA2_512'
2017/10/24 19:38:16 NHRP: VICI: Key 'dh-group'='ECP_384'
2017/10/24 19:38:16 NHRP: VICI: Key 'established'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'rekey-time'='10132'
2017/10/24 19:38:16 NHRP: VICI: Key 'reauth-time'='42389'
2017/10/24 19:38:16 NHRP: VICI: List start 'tasks-active'
2017/10/24 19:38:16 NHRP: VICI: List item: 'CHILD_CREATE'
2017/10/24 19:38:16 NHRP: VICI: List item: 'IKE_AUTH_LIFETIME'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: Section start 'child-sas'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'name'='dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'reqid'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='INSTALLING'
2017/10/24 19:38:16 NHRP: VICI: Key 'mode'='TRANSPORT'
2017/10/24 19:38:16 NHRP: VICI: List start 'local-ts'
2017/10/24 19:38:16 NHRP: VICI: List item: '10.10.10.1/32[gre]'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: List start 'remote-ts'
2017/10/24 19:38:16 NHRP: VICI: List item: '10.10.10.6/32[gre]'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Message 7, 778 bytes
2017/10/24 19:38:16 NHRP: VICI: Event 'child-state-destroying'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='3'
2017/10/24 19:38:16 NHRP: VICI: Key 'version'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='DESTROYING'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-host'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-id'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-host'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-id'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'initiator-spi'='cfec233bea08461c'
2017/10/24 19:38:16 NHRP: VICI: Key 'responder-spi'='4e59549625b7d14e'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-alg'='AES_CBC'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-keysize'='256'
2017/10/24 19:38:16 NHRP: VICI: Key 'integ-alg'='HMAC_SHA2_512_256'
2017/10/24 19:38:16 NHRP: VICI: Key 'prf-alg'='PRF_HMAC_SHA2_512'
2017/10/24 19:38:16 NHRP: VICI: Key 'dh-group'='ECP_384'
2017/10/24 19:38:16 NHRP: VICI: Section start 'child-sas'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'name'='dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='3'
2017/10/24 19:38:16 NHRP: VICI: Key 'reqid'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='INSTALLED'
2017/10/24 19:38:16 NHRP: VICI: Key 'mode'='TRANSPORT'
2017/10/24 19:38:16 NHRP: VICI: Key 'protocol'='ESP'
2017/10/24 19:38:16 NHRP: VICI: Key 'spi-in'='c9210272'
2017/10/24 19:38:16 NHRP: VICI: Key 'spi-out'='c9175927'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-alg'='AES_CBC'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-keysize'='256'
2017/10/24 19:38:16 NHRP: VICI: Key 'integ-alg'='HMAC_SHA2_512_256'
2017/10/24 19:38:16 NHRP: VICI: Key 'bytes-in'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'packets-in'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'bytes-out'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'packets-out'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'rekey-time'='5722'
2017/10/24 19:38:16 NHRP: VICI: Key 'life-time'='6600'
2017/10/24 19:38:16 NHRP: VICI: Key 'install-time'='0'
2017/10/24 19:38:16 NHRP: VICI: List start 'local-ts'
2017/10/24 19:38:16 NHRP: VICI: List item: '10.10.10.1/32[gre]'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: List start 'remote-ts'
2017/10/24 19:38:16 NHRP: VICI: List item: '10.10.10.6/32[gre]'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Message 7, 794 bytes
2017/10/24 19:38:16 NHRP: VICI: Event 'child-state-destroying'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'version'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='DESTROYING'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-host'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-id'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-host'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-id'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'initiator'='yes'
2017/10/24 19:38:16 NHRP: VICI: Key 'initiator-spi'='10576792c380167f'
2017/10/24 19:38:16 NHRP: VICI: Key 'responder-spi'='59b1b0ab0d68784e'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-alg'='AES_CBC'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-keysize'='256'
2017/10/24 19:38:16 NHRP: VICI: Key 'integ-alg'='HMAC_SHA2_512_256'
2017/10/24 19:38:16 NHRP: VICI: Key 'prf-alg'='PRF_HMAC_SHA2_512'
2017/10/24 19:38:16 NHRP: VICI: Key 'dh-group'='ECP_384'
2017/10/24 19:38:16 NHRP: VICI: Section start 'child-sas'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'name'='dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'reqid'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='INSTALLED'
2017/10/24 19:38:16 NHRP: VICI: Key 'mode'='TRANSPORT'
2017/10/24 19:38:16 NHRP: VICI: Key 'protocol'='ESP'
2017/10/24 19:38:16 NHRP: VICI: Key 'spi-in'='c2ddc8b3'
2017/10/24 19:38:16 NHRP: VICI: Key 'spi-out'='c6549711'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-alg'='AES_CBC'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-keysize'='256'
2017/10/24 19:38:16 NHRP: VICI: Key 'integ-alg'='HMAC_SHA2_512_256'
2017/10/24 19:38:16 NHRP: VICI: Key 'bytes-in'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'packets-in'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'bytes-out'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'packets-out'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'rekey-time'='5658'
2017/10/24 19:38:16 NHRP: VICI: Key 'life-time'='6600'
2017/10/24 19:38:16 NHRP: VICI: Key 'install-time'='0'
2017/10/24 19:38:16 NHRP: VICI: List start 'local-ts'
2017/10/24 19:38:16 NHRP: VICI: List item: '10.10.10.1/32[gre]'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: List start 'remote-ts'
2017/10/24 19:38:16 NHRP: VICI: List item: '10.10.10.6/32[gre]'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:17 NHRP: Shortcut 172.31.0.6/32 purged
=========
Spoke1# sh ip nhrp shortcut
Type Prefix Via Identity
cached 172.31.0.0/16 192.168.0.6
Spoke1# sh ip nhrp cache
Iface Type Protocol NBMA Flags Identity
gre1 local 192.168.1.1 - -
gre1 invalid 192.168.0.6 - A -
gre1 nhs 192.168.0.1 10.10.10.5 UT 10.10.10.5
swanctl -l
-- nothing changed
=========
Spoke 2
=========
2017/10/24 19:38:16 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.6
2017/10/24 19:38:16 NHRP: Recv Traffic-Indication(8) 192.168.0.1 -> 172.31.0.6
2017/10/24 19:38:16 NHRP: lookup 172.31.0.6/32: zebra route dev eth1
2017/10/24 19:38:16 NHRP: Traffic Indication from 192.168.0.1 about packet to 10.50.0.1: trying shortcut
2017/10/24 19:38:16 NHRP: Shortcut 10.50.0.1/32 created
2017/10/24 19:38:16 NHRP: lookup 10.50.0.1/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.1.1/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.0.1/32: nhrp_if=gre1
2017/10/24 19:38:16 NHRP: Send Resolution-Request(1) 192.168.0.6 -> 10.50.0.1
2017/10/24 19:38:16 NHRP: PACKET: Send 10.10.10.6 -> 10.10.10.5
2017/10/24 19:38:16 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.6
2017/10/24 19:38:16 NHRP: Recv Resolution-Request(1) 192.168.1.1 -> 172.31.0.6
2017/10/24 19:38:16 NHRP: lookup 172.31.0.6/32: zebra route dev eth1
2017/10/24 19:38:16 NHRP: Parsing and replying to Resolution Req
2017/10/24 19:38:16 NHRP: lookup 192.168.1.1/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.0.1/32: nhrp_if=gre1
2017/10/24 19:38:16 NHRP: Send Resolution-Reply(2) 172.31.0.6 -> 192.168.1.1
2017/10/24 19:38:16 NHRP: PACKET: Send 10.10.10.6 -> 10.10.10.5
2017/10/24 19:38:16 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.6
2017/10/24 19:38:16 NHRP: Recv Resolution-Reply(2) 10.50.0.1 -> 192.168.0.6
2017/10/24 19:38:16 NHRP: !LOCAL Resolution-Reply(2) 10.50.0.1 -> 192.168.0.6
2017/10/24 19:38:16 NHRP: lookup 10.50.0.1/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.1.1/32: zebra route dev (none)
2017/10/24 19:38:16 NHRP: lookup 192.168.0.1/32: nhrp_if=gre1
2017/10/24 19:38:16 NHRP: Shortcut: 10.50.0.1/16 is at proto 192.168.1.1 cie-nbma 10.10.10.1 nat-nbma (unspec) cie-holdtime 7200
2017/10/24 19:38:16 NHRP: Shortcut 10.50.0.0/16 created
2017/10/24 19:38:16 NHRP: Shortcut: Resolution reply handled
2017/10/24 19:38:16 NHRP: VICI: Message 1, 15 bytes
2017/10/24 19:38:16 NHRP: VICI: Key 'success'='yes'
2017/10/24 19:38:16 NHRP: VICI: Message 7, 668 bytes
2017/10/24 19:38:16 NHRP: VICI: Event 'child-state-installed'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='3'
2017/10/24 19:38:16 NHRP: VICI: Key 'version'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='ESTABLISHED'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-host'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-id'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-host'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-id'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'initiator-spi'='10576792c380167f'
2017/10/24 19:38:16 NHRP: VICI: Key 'responder-spi'='59b1b0ab0d68784e'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-alg'='AES_CBC'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-keysize'='256'
2017/10/24 19:38:16 NHRP: VICI: Key 'integ-alg'='HMAC_SHA2_512_256'
2017/10/24 19:38:16 NHRP: VICI: Key 'prf-alg'='PRF_HMAC_SHA2_512'
2017/10/24 19:38:16 NHRP: VICI: Key 'dh-group'='ECP_384'
2017/10/24 19:38:16 NHRP: VICI: Key 'established'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'rekey-time'='13012'
2017/10/24 19:38:16 NHRP: VICI: Key 'reauth-time'='44791'
2017/10/24 19:38:16 NHRP: VICI: List start 'tasks-passive'
2017/10/24 19:38:16 NHRP: VICI: List item: 'CHILD_CREATE'
2017/10/24 19:38:16 NHRP: VICI: List item: 'IKE_AUTH_LIFETIME'
2017/10/24 19:38:16 NHRP: VICI: List item: 'IKE_MOBIKE'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: Section start 'child-sas'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'name'='dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='3'
2017/10/24 19:38:16 NHRP: VICI: Key 'reqid'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='INSTALLING'
2017/10/24 19:38:16 NHRP: VICI: Key 'mode'='TRANSPORT'
2017/10/24 19:38:16 NHRP: VICI: List start 'local-ts'
2017/10/24 19:38:16 NHRP: VICI: List item: '10.10.10.6/32[gre]'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: List start 'remote-ts'
2017/10/24 19:38:16 NHRP: VICI: List item: '10.10.10.1/32[gre]'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Message 7, 576 bytes
2017/10/24 19:38:16 NHRP: VICI: Event 'child-state-destroying'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'version'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='DELETING'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-host'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-id'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-host'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-id'='%any'
2017/10/24 19:38:16 NHRP: VICI: Key 'initiator'='yes'
2017/10/24 19:38:16 NHRP: VICI: Key 'initiator-spi'='cfec233bea08461c'
2017/10/24 19:38:16 NHRP: VICI: Key 'responder-spi'='4e59549625b7d14e'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-alg'='AES_CBC'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-keysize'='256'
2017/10/24 19:38:16 NHRP: VICI: Key 'integ-alg'='HMAC_SHA2_512_256'
2017/10/24 19:38:16 NHRP: VICI: Key 'prf-alg'='PRF_HMAC_SHA2_512'
2017/10/24 19:38:16 NHRP: VICI: Key 'dh-group'='ECP_384'
2017/10/24 19:38:16 NHRP: VICI: List start 'tasks-active'
2017/10/24 19:38:16 NHRP: VICI: List item: 'IKE_CERT_PRE'
2017/10/24 19:38:16 NHRP: VICI: List item: 'IKE_AUTH'
2017/10/24 19:38:16 NHRP: VICI: List item: 'IKE_CERT_POST'
2017/10/24 19:38:16 NHRP: VICI: List item: 'IKE_CONFIG'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: Section start 'child-sas'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'name'='dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'reqid'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='CREATED'
2017/10/24 19:38:16 NHRP: VICI: Key 'mode'='TUNNEL'
2017/10/24 19:38:16 NHRP: VICI: List start 'local-ts'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: List start 'remote-ts'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Message 7, 778 bytes
2017/10/24 19:38:16 NHRP: VICI: Event 'child-state-destroying'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='3'
2017/10/24 19:38:16 NHRP: VICI: Key 'version'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='DESTROYING'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-host'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'local-id'='10.10.10.6'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-host'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-port'='500'
2017/10/24 19:38:16 NHRP: VICI: Key 'remote-id'='10.10.10.1'
2017/10/24 19:38:16 NHRP: VICI: Key 'initiator-spi'='10576792c380167f'
2017/10/24 19:38:16 NHRP: VICI: Key 'responder-spi'='59b1b0ab0d68784e'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-alg'='AES_CBC'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-keysize'='256'
2017/10/24 19:38:16 NHRP: VICI: Key 'integ-alg'='HMAC_SHA2_512_256'
2017/10/24 19:38:16 NHRP: VICI: Key 'prf-alg'='PRF_HMAC_SHA2_512'
2017/10/24 19:38:16 NHRP: VICI: Key 'dh-group'='ECP_384'
2017/10/24 19:38:16 NHRP: VICI: Section start 'child-sas'
2017/10/24 19:38:16 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'name'='dmvpn'
2017/10/24 19:38:16 NHRP: VICI: Key 'uniqueid'='3'
2017/10/24 19:38:16 NHRP: VICI: Key 'reqid'='2'
2017/10/24 19:38:16 NHRP: VICI: Key 'state'='INSTALLED'
2017/10/24 19:38:16 NHRP: VICI: Key 'mode'='TRANSPORT'
2017/10/24 19:38:16 NHRP: VICI: Key 'protocol'='ESP'
2017/10/24 19:38:16 NHRP: VICI: Key 'spi-in'='c6549711'
2017/10/24 19:38:16 NHRP: VICI: Key 'spi-out'='c2ddc8b3'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-alg'='AES_CBC'
2017/10/24 19:38:16 NHRP: VICI: Key 'encr-keysize'='256'
2017/10/24 19:38:16 NHRP: VICI: Key 'integ-alg'='HMAC_SHA2_512_256'
2017/10/24 19:38:16 NHRP: VICI: Key 'bytes-in'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'packets-in'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'bytes-out'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'packets-out'='0'
2017/10/24 19:38:16 NHRP: VICI: Key 'rekey-time'='5602'
2017/10/24 19:38:16 NHRP: VICI: Key 'life-time'='6600'
2017/10/24 19:38:16 NHRP: VICI: Key 'install-time'='0'
2017/10/24 19:38:16 NHRP: VICI: List start 'local-ts'
2017/10/24 19:38:16 NHRP: VICI: List item: '10.10.10.6/32[gre]'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: List start 'remote-ts'
2017/10/24 19:38:16 NHRP: VICI: List item: '10.10.10.1/32[gre]'
2017/10/24 19:38:16 NHRP: VICI: List end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:16 NHRP: VICI: Section end
2017/10/24 19:38:17 NHRP: Shortcut 10.50.0.1/32 purged
2017/10/24 19:39:16 NHRP: cache: gre1 192.168.1.1: timeout
2017/10/24 19:39:16 NHRP: Shortcut 10.50.0.0/16 purged
=========
Spoke2# sh ip nhrp shortcut
Type Prefix Via Identity
cached 10.50.0.0/16 192.168.1.1
Spoke2# sh ip nhrp cache
Iface Type Protocol NBMA Flags Identity
gre1 invalid 192.168.1.1 - A -
gre1 local 192.168.0.6 - -
gre1 nhs 192.168.0.1 10.10.10.5 UT 10.10.10.5
swanctl -l
-- nothing changed
============================
Debug Output from
Registration completed to
tunnel to tunnel ping - WORKS
============================
=========
Hub
=========
<!-- first ping was with a count of 10 pings -->
2017/10/24 19:51:02 NHRP: Netlink-log: Received msg_type 1024, msg_flags 0
2017/10/24 19:51:02 NHRP: lookup 192.168.1.1/32: nhrp_if=gre1
2017/10/24 19:51:02 NHRP: Send Traffic Indication to 10.10.10.1 (online=1) about packet to 192.168.1.1
2017/10/24 19:51:02 NHRP: Send Traffic-Indication(8) 192.168.0.1 -> 192.168.1.1
2017/10/24 19:51:02 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.1
2017/10/24 19:51:02 NHRP: PACKET: Recv 10.10.10.1 -> 10.10.10.5
2017/10/24 19:51:02 NHRP: Recv Resolution-Request(1) 192.168.1.1 -> 192.168.0.6
2017/10/24 19:51:02 NHRP: lookup 192.168.0.6/32: nhrp_if=gre1
2017/10/24 19:51:02 NHRP: Send Resolution-Request(1) 192.168.1.1 -> 192.168.0.6
2017/10/24 19:51:02 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.6
2017/10/24 19:51:02 NHRP: PACKET: Recv 10.10.10.6 -> 10.10.10.5
2017/10/24 19:51:02 NHRP: Recv Resolution-Reply(2) 192.168.0.6 -> 192.168.1.1
2017/10/24 19:51:02 NHRP: lookup 192.168.1.1/32: nhrp_if=gre1
2017/10/24 19:51:02 NHRP: Send Resolution-Reply(2) 192.168.0.6 -> 192.168.1.1
2017/10/24 19:51:02 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.1
<!-- followed by another ping with a count of 20 pings -->
2017/10/24 19:51:25 NHRP: Netlink-log: Received msg_type 1024, msg_flags 0
2017/10/24 19:51:25 NHRP: lookup 192.168.0.6/32: nhrp_if=gre1
2017/10/24 19:51:25 NHRP: Send Traffic Indication to 10.10.10.6 (online=1) about packet to 192.168.0.6
2017/10/24 19:51:25 NHRP: Send Traffic-Indication(8) 192.168.0.1 -> 192.168.0.6
2017/10/24 19:51:25 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.6
2017/10/24 19:51:25 NHRP: PACKET: Recv 10.10.10.6 -> 10.10.10.5
2017/10/24 19:51:25 NHRP: Recv Resolution-Request(1) 192.168.0.6 -> 192.168.1.1
2017/10/24 19:51:25 NHRP: lookup 192.168.1.1/32: nhrp_if=gre1
2017/10/24 19:51:25 NHRP: Send Resolution-Request(1) 192.168.0.6 -> 192.168.1.1
2017/10/24 19:51:25 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.1
2017/10/24 19:51:25 NHRP: PACKET: Recv 10.10.10.1 -> 10.10.10.5
2017/10/24 19:51:25 NHRP: Recv Resolution-Reply(2) 192.168.1.1 -> 192.168.0.6
2017/10/24 19:51:25 NHRP: lookup 192.168.0.6/32: nhrp_if=gre1
2017/10/24 19:51:25 NHRP: Send Resolution-Reply(2) 192.168.1.1 -> 192.168.0.6
2017/10/24 19:51:25 NHRP: PACKET: Send 10.10.10.5 -> 10.10.10.6
=========
Spoke 1
=========
2017/10/24 19:51:02 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.1
2017/10/24 19:51:02 NHRP: Recv Traffic-Indication(8) 192.168.0.1 -> 192.168.1.1
2017/10/24 19:51:02 NHRP: !LOCAL Traffic-Indication(8) 192.168.0.1 -> 192.168.1.1
2017/10/24 19:51:02 NHRP: Traffic Indication from 192.168.0.1 about packet to 192.168.0.6: trying shortcut
2017/10/24 19:51:02 NHRP: Shortcut 192.168.0.6/32 created
2017/10/24 19:51:02 NHRP: lookup 192.168.0.6/32: zebra route dev (none)
2017/10/24 19:51:02 NHRP: lookup 192.168.0.1/32: nhrp_if=gre1
2017/10/24 19:51:02 NHRP: Send Resolution-Request(1) 192.168.1.1 -> 192.168.0.6
2017/10/24 19:51:02 NHRP: PACKET: Send 10.10.10.1 -> 10.10.10.5
2017/10/24 19:51:02 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.1
2017/10/24 19:51:02 NHRP: Recv Resolution-Reply(2) 192.168.0.6 -> 192.168.1.1
2017/10/24 19:51:02 NHRP: !LOCAL Resolution-Reply(2) 192.168.0.6 -> 192.168.1.1
2017/10/24 19:51:02 NHRP: Shortcut: 192.168.0.6/32 is at proto 192.168.0.6 cie-nbma 10.10.10.6 nat-nbma (unspec) cie-holdtime 7200
2017/10/24 19:51:02 NHRP: Shortcut: Resolution reply handled
2017/10/24 19:51:02 NHRP: VICI: Message 1, 15 bytes
2017/10/24 19:51:02 NHRP: VICI: Key 'success'='yes'
2017/10/24 19:51:03 NHRP: VICI: Message 7, 670 bytes
2017/10/24 19:51:03 NHRP: VICI: Event 'child-state-installed'
2017/10/24 19:51:03 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:51:03 NHRP: VICI: Key 'uniqueid'='2'
2017/10/24 19:51:03 NHRP: VICI: Key 'version'='2'
2017/10/24 19:51:03 NHRP: VICI: Key 'state'='ESTABLISHED'
2017/10/24 19:51:03 NHRP: VICI: Key 'local-host'='10.10.10.1'
2017/10/24 19:51:03 NHRP: VICI: Key 'local-port'='500'
2017/10/24 19:51:03 NHRP: VICI: Key 'local-id'='10.10.10.1'
2017/10/24 19:51:03 NHRP: VICI: Key 'remote-host'='10.10.10.6'
2017/10/24 19:51:03 NHRP: VICI: Key 'remote-port'='500'
2017/10/24 19:51:03 NHRP: VICI: Key 'remote-id'='10.10.10.6'
2017/10/24 19:51:03 NHRP: VICI: Key 'initiator'='yes'
2017/10/24 19:51:03 NHRP: VICI: Key 'initiator-spi'='dec145a17d90cdbb'
2017/10/24 19:51:03 NHRP: VICI: Key 'responder-spi'='b9702d9c44edf331'
2017/10/24 19:51:03 NHRP: VICI: Key 'encr-alg'='AES_CBC'
2017/10/24 19:51:03 NHRP: VICI: Key 'encr-keysize'='256'
2017/10/24 19:51:03 NHRP: VICI: Key 'integ-alg'='HMAC_SHA2_512_256'
2017/10/24 19:51:03 NHRP: VICI: Key 'prf-alg'='PRF_HMAC_SHA2_512'
2017/10/24 19:51:03 NHRP: VICI: Key 'dh-group'='ECP_384'
2017/10/24 19:51:03 NHRP: VICI: Key 'established'='0'
2017/10/24 19:51:03 NHRP: VICI: Key 'rekey-time'='13142'
2017/10/24 19:51:03 NHRP: VICI: Key 'reauth-time'='44544'
2017/10/24 19:51:03 NHRP: VICI: List start 'tasks-active'
2017/10/24 19:51:03 NHRP: VICI: List item: 'CHILD_CREATE'
2017/10/24 19:51:03 NHRP: VICI: List item: 'IKE_AUTH_LIFETIME'
2017/10/24 19:51:03 NHRP: VICI: List end
2017/10/24 19:51:03 NHRP: VICI: Section start 'child-sas'
2017/10/24 19:51:03 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:51:03 NHRP: VICI: Key 'name'='dmvpn'
2017/10/24 19:51:03 NHRP: VICI: Key 'uniqueid'='2'
2017/10/24 19:51:03 NHRP: VICI: Key 'reqid'='2'
2017/10/24 19:51:03 NHRP: VICI: Key 'state'='INSTALLING'
2017/10/24 19:51:03 NHRP: VICI: Key 'mode'='TRANSPORT'
2017/10/24 19:51:03 NHRP: VICI: List start 'local-ts'
2017/10/24 19:51:03 NHRP: VICI: List item: '10.10.10.1/32[gre]'
2017/10/24 19:51:03 NHRP: VICI: List end
2017/10/24 19:51:03 NHRP: VICI: List start 'remote-ts'
2017/10/24 19:51:03 NHRP: VICI: List item: '10.10.10.6/32[gre]'
2017/10/24 19:51:03 NHRP: VICI: List end
2017/10/24 19:51:03 NHRP: VICI: Section end
2017/10/24 19:51:03 NHRP: VICI: Section end
2017/10/24 19:51:03 NHRP: VICI: Section end
2017/10/24 19:51:03 NHRP: cache: gre1 192.168.0.6: accept
2017/10/24 19:51:03 NHRP: Zebra send: route add 192.168.0.6/32 nexthop <onlink> metric 0 count 1 dev gre1
2017/10/24 19:51:03 NHRP: Netlink: Received msg_type 28, msg_flags 0
2017/10/24 19:51:03 NHRP: Netlink: update 192.168.0.6 dev gre1 nud 2
2017/10/24 19:51:03 NHRP: if-route-add: 172.31.0.0/16 via 192.168.0.6 dev (none)
2017/10/24 19:51:03 NHRP: Shortcut 192.168.0.6/32 purged
2017/10/24 19:51:13 NHRP: cache: gre1 192.168.0.6: timeout
2017/10/24 19:51:25 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.1
2017/10/24 19:51:25 NHRP: Recv Resolution-Request(1) 192.168.0.6 -> 192.168.1.1
2017/10/24 19:51:25 NHRP: !LOCAL Resolution-Request(1) 192.168.0.6 -> 192.168.1.1
2017/10/24 19:51:25 NHRP: Parsing and replying to Resolution Req
2017/10/24 19:51:25 NHRP: lookup 192.168.0.6/32: zebra route dev (none)
2017/10/24 19:51:25 NHRP: lookup 192.168.0.1/32: nhrp_if=gre1
2017/10/24 19:51:25 NHRP: Send Resolution-Reply(2) 192.168.1.1 -> 192.168.0.6
2017/10/24 19:51:25 NHRP: PACKET: Send 10.10.10.1 -> 10.10.10.5
=========
Spoke1# sh ip nhrp shortcut
% No entries
Spoke1# sh ip nhrp cache
Iface Type Protocol NBMA Flags Identity
gre1 local 192.168.1.1 - -
gre1 cached 192.168.0.6 10.10.10.6 UT 10.10.10.6
gre1 nhs 192.168.0.1 10.10.10.5 UT 10.10.10.5
swanctl -l
-- NEW SA ADDED
Spoke1:/# swanctl -l
dmvpn: #2, ESTABLISHED, IKEv2, dec145a17d90cdbb_i* b9702d9c44edf331_r
local '10.10.10.1' @ 10.10.10.1[500]
remote '10.10.10.6' @ 10.10.10.6[500]
AES_CBC-256/HMAC_SHA2_512_256/PRF_HMAC_SHA2_512/ECP_384
established 285s ago, rekeying in 12857s, reauth in 39271s
dmvpn: #2, reqid 2, INSTALLED, TRANSPORT, ESP:AES_CBC-256/HMAC_SHA2_512_256
installed 285s ago, rekeying in 5525s, expires in 6315s
in c5e9942a, 1748 bytes, 19 packets, 244s ago
out c4676033, 2668 bytes, 29 packets, 243s ago
local 10.10.10.1/32[gre]
remote 10.10.10.6/32[gre]
dmvpn: #1, ESTABLISHED, IKEv2, 142f1c05401517d4_i* 9ca58897716c6484_r
local '10.10.10.1' @ 10.10.10.1[500]
remote '10.10.10.5' @ 10.10.10.5[500]
AES_CBC-256/HMAC_SHA2_512_256/PRF_HMAC_SHA2_512/ECP_384
established 425s ago, rekeying in 10247s, reauth in 41206s
dmvpn: #1, reqid 1, INSTALLED, TRANSPORT, ESP:AES_CBC-256/HMAC_SHA2_512_256
installed 425s ago, rekeying in 5125s, expires in 6175s
in c4212119, 3249 bytes, 36 packets, 4s ago
out c75f1eef, 2295 bytes, 30 packets, 4s ago
local 10.10.10.1/32[gre]
remote 10.10.10.5/32[gre]
Plus New [N] routes added and recursive route to back end network of remote spoke updated
Spoke1# sh ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, P - PIM, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel,
> - selected route, * - FIB route
K>* 0.0.0.0/0 [0/0] is directly connected, eth0, 00:46:13
C>* 10.50.0.0/16 is directly connected, eth1, 00:46:13
C>* 10.10.10.1/32 is directly connected, eth0, 00:46:13
B> 172.31.0.0/16 [200/0] via 192.168.0.6 (recursive), 00:43:50
* via 192.168.0.6, gre1 onlink, 00:43:50
N>* 192.168.0.1/32 [10/0] is directly connected, gre1, 00:46:10
N * 192.168.0.6/32 [10/0] is directly connected, gre1, 00:43:50
B> 192.168.0.6/32 [200/0] via 192.168.0.1 (recursive), 00:46:10
via 192.168.0.1, gre1 onlink, 00:46:10
B 192.168.1.1/32 [200/0] via 192.168.0.1, 00:46:10
C>* 192.168.1.1/32 is directly connected, gre1, 00:46:13
=========
Spoke 2
=========
2017/10/24 19:51:02 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.6
2017/10/24 19:51:02 NHRP: Recv Resolution-Request(1) 192.168.1.1 -> 192.168.0.6
2017/10/24 19:51:02 NHRP: !LOCAL Resolution-Request(1) 192.168.1.1 -> 192.168.0.6
2017/10/24 19:51:02 NHRP: Parsing and replying to Resolution Req
2017/10/24 19:51:02 NHRP: lookup 192.168.1.1/32: zebra route dev (none)
2017/10/24 19:51:02 NHRP: lookup 192.168.0.1/32: nhrp_if=gre1
2017/10/24 19:51:02 NHRP: Send Resolution-Reply(2) 192.168.0.6 -> 192.168.1.1
2017/10/24 19:51:02 NHRP: PACKET: Send 10.10.10.6 -> 10.10.10.5
2017/10/24 19:51:02 NHRP: VICI: Message 7, 668 bytes
2017/10/24 19:51:02 NHRP: VICI: Event 'child-state-installed'
2017/10/24 19:51:02 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:51:02 NHRP: VICI: Key 'uniqueid'='2'
2017/10/24 19:51:02 NHRP: VICI: Key 'version'='2'
2017/10/24 19:51:02 NHRP: VICI: Key 'state'='ESTABLISHED'
2017/10/24 19:51:02 NHRP: VICI: Key 'local-host'='10.10.10.6'
2017/10/24 19:51:02 NHRP: VICI: Key 'local-port'='500'
2017/10/24 19:51:02 NHRP: VICI: Key 'local-id'='10.10.10.6'
2017/10/24 19:51:02 NHRP: VICI: Key 'remote-host'='10.10.10.1'
2017/10/24 19:51:02 NHRP: VICI: Key 'remote-port'='500'
2017/10/24 19:51:02 NHRP: VICI: Key 'remote-id'='10.10.10.1'
2017/10/24 19:51:02 NHRP: VICI: Key 'initiator-spi'='dec145a17d90cdbb'
2017/10/24 19:51:02 NHRP: VICI: Key 'responder-spi'='b9702d9c44edf331'
2017/10/24 19:51:02 NHRP: VICI: Key 'encr-alg'='AES_CBC'
2017/10/24 19:51:02 NHRP: VICI: Key 'encr-keysize'='256'
2017/10/24 19:51:02 NHRP: VICI: Key 'integ-alg'='HMAC_SHA2_512_256'
2017/10/24 19:51:02 NHRP: VICI: Key 'prf-alg'='PRF_HMAC_SHA2_512'
2017/10/24 19:51:02 NHRP: VICI: Key 'dh-group'='ECP_384'
2017/10/24 19:51:02 NHRP: VICI: Key 'established'='0'
2017/10/24 19:51:02 NHRP: VICI: Key 'rekey-time'='10490'
2017/10/24 19:51:02 NHRP: VICI: Key 'reauth-time'='44236'
2017/10/24 19:51:02 NHRP: VICI: List start 'tasks-passive'
2017/10/24 19:51:02 NHRP: VICI: List item: 'CHILD_CREATE'
2017/10/24 19:51:02 NHRP: VICI: List item: 'IKE_AUTH_LIFETIME'
2017/10/24 19:51:02 NHRP: VICI: List item: 'IKE_MOBIKE'
2017/10/24 19:51:02 NHRP: VICI: List end
2017/10/24 19:51:02 NHRP: VICI: Section start 'child-sas'
2017/10/24 19:51:02 NHRP: VICI: Section start 'dmvpn'
2017/10/24 19:51:02 NHRP: VICI: Key 'name'='dmvpn'
2017/10/24 19:51:02 NHRP: VICI: Key 'uniqueid'='2'
2017/10/24 19:51:02 NHRP: VICI: Key 'reqid'='2'
2017/10/24 19:51:02 NHRP: VICI: Key 'state'='INSTALLING'
2017/10/24 19:51:02 NHRP: VICI: Key 'mode'='TRANSPORT'
2017/10/24 19:51:02 NHRP: VICI: List start 'local-ts'
2017/10/24 19:51:02 NHRP: VICI: List item: '10.10.10.6/32[gre]'
2017/10/24 19:51:02 NHRP: VICI: List end
2017/10/24 19:51:02 NHRP: VICI: List start 'remote-ts'
2017/10/24 19:51:02 NHRP: VICI: List item: '10.10.10.1/32[gre]'
2017/10/24 19:51:02 NHRP: VICI: List end
2017/10/24 19:51:02 NHRP: VICI: Section end
2017/10/24 19:51:02 NHRP: VICI: Section end
2017/10/24 19:51:02 NHRP: VICI: Section end
2017/10/24 19:51:25 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.6
2017/10/24 19:51:25 NHRP: Recv Traffic-Indication(8) 192.168.0.1 -> 192.168.0.6
2017/10/24 19:51:25 NHRP: !LOCAL Traffic-Indication(8) 192.168.0.1 -> 192.168.0.6
2017/10/24 19:51:25 NHRP: Traffic Indication from 192.168.0.1 about packet to 192.168.1.1: trying shortcut
2017/10/24 19:51:25 NHRP: Shortcut 192.168.1.1/32 created
2017/10/24 19:51:25 NHRP: lookup 192.168.1.1/32: zebra route dev (none)
2017/10/24 19:51:25 NHRP: lookup 192.168.0.1/32: nhrp_if=gre1
2017/10/24 19:51:25 NHRP: Send Resolution-Request(1) 192.168.0.6 -> 192.168.1.1
2017/10/24 19:51:25 NHRP: PACKET: Send 10.10.10.6 -> 10.10.10.5
2017/10/24 19:51:25 NHRP: PACKET: Recv 10.10.10.5 -> 10.10.10.6
2017/10/24 19:51:25 NHRP: Recv Resolution-Reply(2) 192.168.1.1 -> 192.168.0.6
2017/10/24 19:51:25 NHRP: !LOCAL Resolution-Reply(2) 192.168.1.1 -> 192.168.0.6
2017/10/24 19:51:25 NHRP: Shortcut: 192.168.1.1/32 is at proto 192.168.1.1 cie-nbma 10.10.10.1 nat-nbma (unspec) cie-holdtime 7200
2017/10/24 19:51:25 NHRP: cache: gre1 192.168.1.1: accept
2017/10/24 19:51:25 NHRP: Zebra send: route add 192.168.1.1/32 nexthop <onlink> metric 0 count 1 dev gre1
2017/10/24 19:51:25 NHRP: Shortcut: Resolution reply handled
2017/10/24 19:51:25 NHRP: Netlink: Received msg_type 28, msg_flags 0
2017/10/24 19:51:25 NHRP: Netlink: update 192.168.1.1 dev gre1 nud 2
2017/10/24 19:51:26 NHRP: if-route-add: 10.50.0.0/16 via 192.168.1.1 dev (none)
2017/10/24 19:51:26 NHRP: Shortcut 192.168.1.1/32 purged
========================
Spoke2# sh ip nhrp shortcut
% No entries
Spoke2# sh ip nhrp cache
Iface Type Protocol NBMA Flags Identity
gre1 cached 192.168.1.1 10.10.10.1 UT 10.10.10.1
gre1 local 192.168.0.6 - -
gre1 nhs 192.168.0.1 10.10.10.5 UT 10.10.10.5
swanctl -l
-- NEW SA ADDED
Spoke2:/# swanctl -l
dmvpn: #2, ESTABLISHED, IKEv2, dec145a17d90cdbb_i b9702d9c44edf331_r*
local '10.10.10.6' @ 10.10.10.6[500]
remote '10.10.10.1' @ 10.10.10.1[500]
AES_CBC-256/HMAC_SHA2_512_256/PRF_HMAC_SHA2_512/ECP_384
established 526s ago, rekeying in 9964s, reauth in 43710s
dmvpn: #2, reqid 2, INSTALLED, TRANSPORT, ESP:AES_CBC-256/HMAC_SHA2_512_256
installed 526s ago, rekeying in 5141s, expires in 6074s
in c4676033, 2668 bytes, 29 packets, 485s ago
out c5e9942a, 1748 bytes, 19 packets, 484s ago
local 10.10.10.6/32[gre]
remote 10.10.10.1/32[gre]
dmvpn: #1, ESTABLISHED, IKEv2, a743e69aa6353864_i* 69d241d7037086ae_r
local '10.10.10.6' @ 10.10.10.6[500]
remote '10.10.10.5' @ 10.10.10.5[500]
AES_CBC-256/HMAC_SHA2_512_256/PRF_HMAC_SHA2_512/ECP_384
established 682s ago, rekeying in 10767s, reauth in 38804s
dmvpn: #1, reqid 1, INSTALLED, TRANSPORT, ESP:AES_CBC-256/HMAC_SHA2_512_256
installed 682s ago, rekeying in 4797s, expires in 5918s
in ce60b554, 2825 bytes, 30 packets, 22s ago
out c2a6b8ad, 3951 bytes, 51 packets, 22s ago
local 10.10.10.6/32[gre]
remote 10.10.10.5/32[gre]
Plus New [N] routes added and recursive route to back end network of remote spoke updated
Spoke2# sh ip route
Codes: K - kernel route, C - connected, S - static, R - RIP,
O - OSPF, I - IS-IS, B - BGP, P - PIM, E - EIGRP, N - NHRP,
T - Table, v - VNC, V - VNC-Direct, A - Babel,
> - selected route, * - FIB route
K>* 0.0.0.0/0 [0/0] is directly connected, eth0, 00:48:21
B> 10.50.0.0/16 [200/0] via 192.168.1.1 (recursive), 00:45:19
* via 192.168.1.1, gre1 onlink, 00:45:19
C>* 10.10.10.6/32 is directly connected, eth0, 00:48:21
C>* 172.31.0.0/16 is directly connected, eth1, 00:48:21
N>* 192.168.0.1/32 [10/0] is directly connected, gre1, 00:48:19
B 192.168.0.6/32 [200/0] via 192.168.0.1, 00:48:19
C>* 192.168.0.6/32 is directly connected, gre1, 00:48:21
N * 192.168.1.1/32 [10/0] is directly connected, gre1, 00:45:20
B> 192.168.1.1/32 [200/0] via 192.168.0.1 (recursive), 00:48:02
via 192.168.0.1, gre1 onlink, 00:48:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment