DDOS protection with iptable

gistfile1.txt

IPtables DDOS protection :

In my config i assume the server is not a router and already profit from some filtering by the hosting company on shitty 
networks.

I have tested this on ubuntu server 18.04 with the kernel 4.15.0-36-generic.

Protect from malformed packet, ACK FIN RST attack and SYN-flood.
Flood which profit of TCP-KEEPALIVE (so there a no SYN packet) should be handled by the web server (rate-limit in nginx for
exemple). Connlimit can also be helpfull to limit the number of connexion per ip.
Protection from bogus network and private network can be added.

SYNPROXY is very helpful to handle synflood but it act as a proxy for every SYN request so it add extra time to establish new
legitimate connexion. You can mitigate synflood with a more basic hashlimit rule per IP or /24 mask but if the attacker is 
using spoofed ip for synflood you will possibly ban good ip. example of rule :

#Rate limit SYN packet per block of 256 IP
iptables -t mangle -A PREROUTING -p tcp --syn -m hashlimit \
--hashlimit-above 200/sec \
--hashlimit-burst 10 \
--hashlimit-mode srcip \
--hashlimit-srcmask 24 \
--hashlimit-name hl_syn \
-j SET --add-set hashlimit_blacklist src # OR -j DROP


If you host services who open multiple port like FTP consider adding the RELATED state to this rule : (check the iptables 
connection tracking helper)
# Accept all established inbound connections
iptables -t filter -A INPUT -m conntrack —cstate ESTABLISHED -j ACCEPT

Some kernel tunning may depend on the ressource available on your server (RAM, L3 cache ...), but keep in mind that the amount 
of traffic you will receive will first depend on the bandwith capacity of your network, so there is probably no need to put 
very high values except if you have 1 or 10 Gbps NIC with the according internet bandwith.

BE CAREFUL WITH FILTERING ON THE PREROUTING CHAIN, it match the packet earlier than in the INPUT chain
so it is more efficient but it match incomming AND outgoing traffic.

DOC : 
-----------------------------------------------------------------------------------------------------------------
https://www.kernel.org/doc/Documentation/networking/nf_conntrack-sysctl.txt
https://www.frozentux.net/ipsysctl-tutorial/ipsysctl-tutorial.html
https://people.netfilter.org/hawk/presentations/dknog4/iptables-ddos-mitigation_JesperBrouer.pdf
https://www.blackhat.com/docs/eu-15/materials/eu-15-Majkowski-Lessons-From-Defending-The-Indefensible.pdf
https://geekeries.org/2017/12/configuration-avancee-du-firewall-iptables/
https://javapipe.com/ddos/blog/iptables-ddos-protection/
https://wiki.mikejung.biz/Sysctl_tweaks
https://home.regit.org/netfilter-en/secure-use-of-helpers/
https://rhelblog.redhat.com/2014/04/11/mitigate-tcp-syn-flood-attacks-with-red-hat-enterprise-linux-7-beta/

KERNEL TUNING :
-----------------------------------------------------------------------------------------------------------------
FIRST:
add "ip_conntrack" and "tcp_bbr" to /etc/modules to charge the modules automatically at boot.
install ipset and conntrack

# Enable Spoof protection (reverse-path filter) Turn on Source Address Verification in all interfaces to prevent some spoofing
attacks
net.ipv4.conf.default.rp_filter=1 #default 1
net.ipv4.conf.all.rp_filter=1 # default 1
# disable routing
net.ipv4.ip_forward =0 # default 0
# Do not accept ICMP redirects (prevent some MITM attacks)
net.ipv4.conf.all.accept_redirects=0 # default 1
net.ipv4.conf.all.secure_redirects=0 # default 1
net.ipv4.conf.default.accept_redirects=0 # default 1
net.ipv4.conf.default.secure_redirects=0 # default 1
net.ipv6.conf.all.accept_redirects=0 # default 1
net.ipv6.conf.default.accept_redirects=0 # default 1
# Accept ICMP redirects only for gateways listed in our default gateway list
net.ipv4.conf.all.secure_redirects=1 # default 1
# Do not send ICMP redirects (we are not a router)
net.ipv4.conf.all.send_redirects=0 # default 1
net.ipv4.conf.default.send_redirects=0 # default 1
# Do not accept IP source route packets (we are not a router)
net.ipv4.conf.all.accept_source_route=0 # default 0
net.ipv4.conf.default.accept_source_route=0 # default 1
net.ipv6.conf.all.accept_source_route=0 # default 0
net.ipv6.conf.default.accept_source_route=0 # default 0
# Log packet with unusual ip
net.ipv4.conf.all.log_martians=1 # default 0
# RFC 1337
net.ipv4.tcp_rfc1337=1 # default 0
#Ignore invalid respsonse as RFC 1122
net.ipv4.icmp_ignore_bogus_error_responses=1 # default 1
#Extend the number of ephemeral port
net.ipv4.ip_local_port_range=32768 65535 # default 32768	60999
#Deactivate the router solicitations
net.ipv6.conf.all.router_solicitations=0 # default 3
net.ipv6.conf.default.router_solicitations=0 # default 3
#Do not accept router preferences from router advertisements
net.ipv6.conf.all.accept_ra_rtr_pref=0 # default 1
net.ipv6.conf.default.accept_ra_rtr_pref=0 # default 1
#Disable auto configuration of prefix from router advertisements
net.ipv6.conf.all.accept_ra_pinfo=0 # default 1
net.ipv6.conf.default.accept_ra_pinfo=0 # default 1
#No default router from router advertisements
net.ipv6.conf.all.accept_ra_defrtr=0 # default 1
net.ipv6.conf.default.accept_ra_defrtr=0 # default 1
#Disable auto configuration of address frorm router advertisements
net.ipv6.conf.all.autoconf=0 # default 1
net.ipv6.conf.default.autoconf=0 # default 1
#Max number of auto configured address per interface
net.ipv6.conf.all.max_addresses=1 # default 16
net.ipv6.conf.default.max_addresses=1 # default 16

# Discourage Linux from swapping out idle processes to disk
vm.swappiness=20 # default 60
vm.dirty_ratio=40 # default 20
vm.dirty_background_ratio=10 # default 10

#System-wide file descriptors limits
fs.file-max=300000 # default 97032
#Maximum parrallel opened sockets that the kernel will server at one time.
net.core.somaxconn=65535 # default 128

#Maximal number of remembered connection requests, which still did not receive an acknowledgement from connecting client.
net.ipv4.tcp_max_syn_backlog=65535 # default 128
#Enable the use of syncookies when the syn backlog queue is full.
net.ipv4.tcp_syncookies=1 # default 1
#Tells the kernel to use timestamps as defined in RFC 1323.
net.ipv4.tcp_timestamps=1 # default 1
#How many times to retransmit the SYN,ACK reply to an SYN request.
net.ipv4.tcp_synack_retries=1 # default 5
#How many TCP sockets that are not attached to any user file handle to maintain.
net.ipv4.tcp_max_orphans=32768 # default 4096
#How many times to retry to kill connections on the other side before killing it on our own side.
net.ipv4.tcp_orphan_retries=1 # default 0
#Maximum number of sockets in TIME-WAIT to be held simultaneously
net.ipv4.tcp_max_tw_buckets=360000 # default 4096
#Allow reusing sockets in TIME_WAIT state for new connections
net.ipv4.tcp_tw_reuse=1 # default 0
#How long to keep sockets in the state FIN-WAIT-2 if you were the one closing the socket.
net.ipv4.tcp_fin_timeout=5 # default 60
#Use TCP BBR to improve tcp congestion control
#net.ipv4.tcp_congestion_control=bbr #default reno cubic

#Disable picking up already established connections.
net.netfilter.nf_conntrack_tcp_loose=0
#Desactivate the automatic conntrack helper assignment.
net.netfilter.nf_conntrack_helper=0
#Size of connection tracking table
net.netfilter.nf_conntrack_max=65536
#Size of hash table
net.netfilter.nf_conntrack_buckets=16384

IPTABLES RULES :
-----------------------------------------------------------------------------------------------------------------

### [ START ] ###

# Reset all rules
iptables -t filter -F 
iptables -t filter -X 
iptables -t nat -F 
iptables -t nat -X 
iptables -t mangle -F
iptables -t mangle -X
iptables -t raw -F
iptables -t raw -X

# Block all INPUT OUTPUT and FORWARD packet by default
iptables -t filter -P INPUT DROP
iptables -t filter -P FORWARD DROP
iptables -t filter -P OUTPUT DROP

#Create the connlimit_blacklist ipset list
ipset -exist create connlimit_blacklist hash:net timeout 20
#Create the hashlimit_blacklist ipset list
ipset -exist create hashlimit_blacklist hash:net timeout 60

#create the chain LOG and hashlimit
$IPT -N LOG_HASH
$IPT -A LOG_HASH -j LOG --log-prefix "RAW:HASHLIMIT_BLACKLIST:" --log-level 4 #(KERN_WARNING)
$IPT -A LOG_HASH -j SET --add-set hashlimit_blacklist src
#create the chain LOG and connlimit
$IPT -N LOG_CONN
$IPT -A LOG_CONN -j LOG --log-prefix "MANGLE:CONNLIMIT_BLACKLIST:" --log-level 4 #(KERN_WARNING)
$IPT -A LOG_CONN -j SET --add-set connlimit_blacklist src


# Allow all loopback (lo0) traffic
iptables -t filter -A INPUT -i lo -j ACCEPT
iptables -t filter -A OUTPUT -o lo -j ACCEPT

### [ PREROUTING ] ###


#hashlimit blacklist
iptables -t raw -A PREROUTING -m set --match-set hashlimit_blacklist src -j DROP
#connlimit blacklist
iptables -t mangle -A PREROUTING -p tcp --syn -m set --match-set connlimit_blacklist src -j DROP

# Do not track SYN packet to :80 for SYNFLOOD
iptables -t raw -A PREROUTING -p tcp -m tcp --dport 80 --syn -j CT --notrack

#Ratelimit the ACK from 3WHS handled by SYNPROXY
iptables -t mangle -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK ACK -m conntrack --ctstate INVALID -m hashlimit \
--hashlimit-above 200/sec \
--hashlimit-burst 10 \
--hashlimit-mode srcip \
--hashlimit-srcmask 24 \
--hashlimit-name hl_syn \
-j LOG_HASH

#Block new packet that are not SYN
iptables -t mangle -A PREROUTING -p tcp ! --syn -m conntrack --ctstate NEW -j DROP
#Block new packet with uncommon MSS value
iptables -t mangle -A PREROUTING -p tcp -m conntrack --ctstate NEW -m tcpmss ! --mss 536:65535 -j DROP
#Block XMAS packet
iptables -t mangle -A PREROUTING -p tcp -m tcp --tcp-flags ALL ALL -j DROP 
#Block NULL packet
iptables -t mangle -A PREROUTING -p tcp -m tcp --tcp-flags ALL NONE -j DROP 
#Block SYN and FIN at the same time
iptables -t mangle -A PREROUTING -p tcp -m tcp --tcp-flags FIN,SYN FIN,SYN -j DROP
#Block SYN and RESET at the same time
iptables -t mangle -A PREROUTING -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
#Block FIN and RESET at the same time
iptables -t mangle -A PREROUTING -p tcp -m tcp --tcp-flags FIN,RST FIN,RST -j DROP
#Block FIN without ACK
iptables -t mangle -A PREROUTING -p tcp -m tcp --tcp-flags FIN,ACK FIN -j DROP
#Block URG without ACK
iptables -t mangle -A PREROUTING -p tcp -m tcp --tcp-flags ACK,URG URG -j DROP
#Block PSH without ACK
iptables -t mangle -A PREROUTING -p tcp -m tcp --tcp-flags PSH,ACK PSH -j DROP

### [ INPUT ] ###

# Accept all established inbound connections
iptables -t filter -A INPUT -m conntrack --ctstate ESTABLISHED -j ACCEPT

#Redirect SYN and ACK for new connection to SYNPROXY
iptables -t filter -A INPUT -p tcp -m tcp --dport 80 -m conntrack --ctstate INVALID,UNTRACKED -j SYNPROXY \
--sack-perm --timestamp --wscale 7 --mss 1460

#Drop rest of state INVALID
iptables -t filter -A INPUT -p tcp -m tcp --dport 80 -m conntrack --ctstate INVALID -j DROP

#Limit to 20 concurrent connection per IP to port 80
iptables -t filter -A INPUT -p tcp --syn --dport 80 -m connlimit \
--connlimit-above 20 \
--connlimit-mask 32 \
--connlimit-saddr \
 -j LOG_CONN

#Accept new packet on port 80
iptables  -t filter -A INPUT -p tcp --dport 80 -m conntrack --ctstate NEW  -j ACCEPT

### [ OUTPUT ] ###

# Accept all established and related outbound connections
iptables -t filter -A OUTPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

#Allow the response to the SYN for the 3WHS before the connection is marked as established 
iptables -t filter -A OUTPUT -p tcp --sport 80 -m tcp --tcp-flags ALL ACK,SYN -j ACCEPT

#Log output dropped packet
iptables -A OUTPUT -m limit --limit 1/second --limit-burst 5 -j LOG --log-prefix "OUTPUT:DROP:" --log-level 4 #(KERN_WARNING)

DEBUG :
--------------------------------------------------------------------------------------------------------

to show the sysctl parameter applied : 
sysctl --system
just use "sysctl" to see the value of a key and to change it
to load a module : 
modprobe -a
to unload a module :
modprobe -r 

ipset : 
ipset list
ipset flush

What about the iptables rules for IPv6?