Skip to content

Instantly share code, notes, and snippets.

@bahorn

bahorn/BGPNotes.md

Last active July 10, 2021 00:28
Show Gist options
  • Save bahorn/b38d6e8844dbc9773064df4aa0df06e7 to your computer and use it in GitHub Desktop.
Save bahorn/b38d6e8844dbc9773064df4aa0df06e7 to your computer and use it in GitHub Desktop.
Download ZIP
Playing with BGP
Raw
BGPNotes.md

BGP with RPKI on KVM

Hacky setup to build a test BGP network with RPKI enabled on it.

Written from memory, does disable some security things, Not how you'd do this in production, etc.

Routers

Two Vyos Routers:

  • router one with AS64512, on 192.168.100.1
  • router two with AS64513, on 192.168.100.2

Router one has three interfaces:

  • eth0 for NAT'd internet access
  • eth1 for BGP
  • eth2 for Clients (DHCP)

Router two has two interfaces:

  • eth0 for BGP
  • eth1 for Client (DHCP)

RPKI Server

Ubuntu VM running:

Krill

Do the initial setup so a configuration file is generated, and then put it into testbed mode by adding the following to /etc/krill.conf:

[testbed]
rrdp_base_uri = "https://localhost:3000/rrdp/"
rsync_jail = "rsync://localhost/repo/"
ta_aia = "rsync://localhost/ta/ta.cer"
ta_uri = "https://localhost:3000/ta/ta.cer"

Also, to save your browser from freezing set bgp_risdumps_enabled = false further up in the configuration.

Restart it, and you should now have a testbed trust anchor at https://localhost:3000/ta/ta.tal, otherwise wipe /var/lib/krill.

You can now configure your routes there so they'll be accept by your routers.

Rsync

Following copies the trust anchor to a suitable place:

mkdir /srv/ta
wget --no-check-certificate https://localhost:3000/ta/ta.tal -O /srv/ta/ta.tal
wget --no-check-certificate https://localhost:3000/ta/ta.cer -O /srv/ta/ta.cer

Patch /srv/ta/ta.tal to include an extra line below the first with the rsync url to get it.

e.g:

rsync://localhost/ta/ta.cer

Then add the following to /etc/rsync.conf:

uid = nobody
gid = nogroup
max connections = 50
socket options = SO_KEEPALIVE

[ta]
path = /srv/ta
comment = RPKI info
read only = yes

[repo]
path = /var/lib/krill/data/repo/rsync/current/
comment = RPKI repository
read only = yes

Then you can systemctl start rsync.service and it should be up.

Routininator

Finally, getting routinator up.

Can just install it from docker, cargo or as a debian package.

Only thing we need to do is replace the tals to just contain our one. So init it, wipe the cache then copy /srv/ta/ta.tal into the directory with them, and remove the old ones so they don't get queried.

Networking

In KVM, create three isolated networks:

  • internal - 192.168.100.0/24, No DHCP
  • lan-1 - 192.168.150.0/24, No DHCP
  • lan-2 - 192.168.160.0/24, No DHCP.

The first can have DHCP, just makes configuring a pain, other two shouldn't has as it's managed by the routers.

  • Router one is on internal and lan-1, router two is on internal and lan-2.
  • Put devices on whatever lan makes sense.
  • RPKI server goes on both lan-1 and lan-2
Raw
fake-route.sh
set protocols static route 1.3.3.7/32 blackhole distance '254'
set protocols bgp address-family ipv4-unicast network '1.3.3.7/32'
set interfaces dummy dum0 address 1.3.3.7/32
Raw
nat.sh
# Router one
## NAT
set nat source rule 100 outbound-interface 'eth0'
set nat source rule 100 source address '192.168.150.0/24'
set nat source rule 100 translation address 'masquerade'
set nat source rule 110 outbound-interface 'eth0'
set nat source rule 110 source address '192.168.150.0/24'
set nat source rule 110 translation address 'masquerade'
## Set the default route to whatever is upstream, 192.168.1.1 in this example
set protocols static route 0.0.0.0/0 next-hop 192.168.1.1
# router two
## Just set a default route by the main one.
set protocols static route 0.0.0.0/0 next-hop 192.168.100.1
Raw
router-1.sh
# Avoiding an issue with the DHCP server not starting if it can't touch /config/dhcp.leases
sudo mkdir /config
configure
# Setup Users
set system login user a full-name a
set system login user a authentication public-keys a@host type ssh-rsa
set system login user a authentication public-keys a@host key "RSA_KEY_HERE"
# Setup Interfaces
## Uplink
set interfaces ethernet eth0 address dhcp
set interfaces ethernet eth0 description 'UPLINK'
## BGP Interfaces
set interfaces ethernet eth1 address 192.168.100.1
set interfaces ethernet eth1 description 'BGP'
## Client Interfaces
set interfaces ethernet eth2 address 192.168.150.1/24
set interfaces ethernet eth2 description 'CLIENTS'
## DNS
set system name-server 8.8.8.8
set system name-server 8.8.4.4
# Network Services
## SSH
set service ssh port 22
set service ssh disable-password-authentication
## Client DNS
set service dns forwarding system
set service dns forwarding name-server 8.8.8.8
set service dns forwarding name-server 8.8.4.4
set service dns forwarding listen-address 192.168.150.1
set service dns forwarding allow-from 0.0.0.0/0
## DHCP
set service dhcp-server shared-network-name NET-lan authoritative
set service dhcp-server shared-network-name NET-lan subnet 192.168.150.0/24 default-router 192.168.150.1
set service dhcp-server shared-network-name NET-lan subnet 192.168.150.0/24 dns-server 192.168.150.1
set service dhcp-server shared-network-name NET-lan subnet 192.168.150.0/24 lease 86400
set service dhcp-server shared-network-name NET-lan subnet 192.168.150.0/24 range 0 start 192.168.150.100
set service dhcp-server shared-network-name NET-lan subnet 192.168.150.0/24 range 0 stop 192.168.150.200
## BGP
set protocols bgp local-as 64512
set protocols bgp neighbor 192.168.100.2 ebgp-multihop '2'
set protocols bgp neighbor 192.168.100.2 remote-as '64513'
set protocols bgp neighbor 192.168.100.2 update-source '192.168.100.1'
set protocols bgp address-family ipv4-unicast network '192.168.150.0/24'
set protocols bgp parameters router-id '192.168.100.1'
## RPKI
set protocols rpki cache 192.168.150.100 port '3323'
set protocols rpki cache 192.168.150.100 preference '1'
set protocols rpki polling-period 10
set policy route-map ROUTES-IN rule 10 action 'permit'
set policy route-map ROUTES-IN rule 10 match rpki 'valid'
set policy route-map ROUTES-IN rule 10 set local-preference '300'
set policy route-map ROUTES-IN rule 20 action 'deny'
set policy route-map ROUTES-IN rule 20 match rpki 'notfound'
set policy route-map ROUTES-IN rule 30 action 'deny'
set policy route-map ROUTES-IN rule 30 match rpki 'invalid'
set protocols bgp neighbor 192.168.100.2 address-family ipv4-unicast route-map import 'ROUTES-IN'
set protocols bgp neighbor 192.168.100.2 address-family ipv4-unicast route-map export 'ROUTES-IN'
commit
save
Raw
router-2.sh
# Avoiding an issue with the DHCP server not starting if it can't touch /config/dhcp.leases
sudo mkdir /config
configure
# Setup Users
set system login user a full-name a
set system login user a authentication public-keys a@host type ssh-rsa
set system login user a authentication public-keys a@host key "RSA_KEY_HERE"
# Setup Interfaces
## Uplink
set interfaces ethernet eth0 address 192.168.100.2
set interfaces ethernet eth0 description 'BGP'
## Client Interfaces
set interfaces ethernet eth1 address 192.168.160.1/24
set interfaces ethernet eth1 description 'CLIENTS'
## DNS
set system name-server 8.8.8.8
set system name-server 8.8.4.4
# Network Services
## SSH
set service ssh port 22
set service ssh disable-password-authentication
## Client DNS
set service dns forwarding system
set service dns forwarding name-server 192.168.160.1
set service dns forwarding name-server 8.8.4.4
set service dns forwarding listen-address 192.168.160.1
set service dns forwarding allow-from 0.0.0.0/0
## DHCP
set service dhcp-server shared-network-name NET-lan authoritative
set service dhcp-server shared-network-name NET-lan subnet 192.168.160.0/24 default-router 192.168.160.1
set service dhcp-server shared-network-name NET-lan subnet 192.168.160.0/24 dns-server 192.168.160.1
set service dhcp-server shared-network-name NET-lan subnet 192.168.160.0/24 lease 86400
set service dhcp-server shared-network-name NET-lan subnet 192.168.160.0/24 range 0 start 192.168.160.100
set service dhcp-server shared-network-name NET-lan subnet 192.168.160.0/24 range 0 stop 192.168.160.200
## BGP
set protocols bgp local-as 64513
set protocols bgp neighbor 192.168.100.1 ebgp-multihop '2'
set protocols bgp neighbor 192.168.100.1 remote-as '64512'
set protocols bgp neighbor 192.168.100.1 update-source '192.168.100.2'
set protocols bgp address-family ipv4-unicast network '192.168.160.0/24'
set protocols bgp parameters router-id '192.168.100.2'
## RPKI
set protocols rpki cache 192.168.160.100 port '3323'
set protocols rpki cache 192.168.160.100 preference '1'
set protocols rpki polling-period 10
set policy route-map ROUTES-IN rule 10 action 'permit'
set policy route-map ROUTES-IN rule 10 match rpki 'valid'
set policy route-map ROUTES-IN rule 10 set local-preference '300'
set policy route-map ROUTES-IN rule 20 action 'deny'
set policy route-map ROUTES-IN rule 20 match rpki 'notfound'
set policy route-map ROUTES-IN rule 30 action 'deny'
set policy route-map ROUTES-IN rule 30 match rpki 'invalid'
set protocols bgp neighbor 192.168.100.1 address-family ipv4-unicast route-map import 'ROUTES-IN'
set protocols bgp neighbor 192.168.100.1 address-family ipv4-unicast route-map export 'ROUTES-IN'
commit
save
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment