🇬🇧 𝗢𝗽𝗲𝗻𝗪𝗿𝘁, 𝗺𝘄𝗮𝗻𝟯 𝗮𝗻𝗱 𝗱𝗲𝗳𝗮𝘂𝗹𝘁 𝗿𝗼𝘂𝘁𝗲 𝗳𝗼𝗿 𝗜𝗣𝘀𝗲𝗰 𝘁𝘂𝗻𝗻𝗲𝗹
OpenWrt mwan3 and IPsec failover: Resolve default route issues for seamless internet & VPN redundancy. Learn how to configure mwan3.user for automatic metric adjustments and IPsec tunnel switching.
https://dariusz.wieckiewicz.org/en/openwrt-mwan3-default-route-ipsec/
#OpenWrt
#ipsec
#wireguard
#mwan3
#defaultroute
#iproute
#failover
#linux #routing #iproute2 #netfilter #iproute #network #networking
for better understanding of this article i suggest to read https://social.dark-alexandr.net/notice/9s2eXIBlOLNF5BIgRE this first because i will omit info explained where.
assuming what you have read mentioned earlier article and understand basics of multiple routing table and ip rule command.
goals:
implementation:
let’s define some variables:
uplink1_device = tap0 uplink1_gw_address = 10.0.0.1 uplink1_own_address = 10.0.0.2 uplink2_device = tap1 uplink2_gw_address = 10.0.1.1 uplink2_own_address = 10.0.1.2
multipath_table_name = multipath multipath_table_num = 10 multipath_table_mark = 0xa
Dariusz Więckiewicz 
Neustradamus :xmpp: :linux:
Linuxiarze
Quentin Demouliere
OSTechNix
sss10 multipathand we will use 0xa mark for this table
ip route add default table $multipath_table_num nexthop via $uplink1_gw_address dev $uplink1_device weight 1 onlink nexthop via $uplink1_gw_address dev $uplink1_device weight 2 onlinkweight is hop priority, more is higher
ip rule add from all fwmark $multipath_table_mark lookup $multipath_table_name#clear existing ruleset
flush ruleset
#define variables (nftables do support variables which is very useful)
define uplink1_dev = "tap0"
define uplink2_dev = "tap1"
define uplink1_local_ip = 10.66.6.13
define uplink2_local_ip = 10.66.13.13
define uplink_devs = { $uplink1_dev,$uplink2_dev}
define uplink_local_ips = { $uplink1_local_ip, $uplink2_local_ip}
define uplink1_mark = 0x1
define uplink2_mark = 0x2
define multipath_mark = 0xa
table mangle {
chain output {
type route hook output priority -150
policy accept
#this is already marked connection already associated with some uplink, let packet go, do not need any handling
ct mark $uplink1_mark counter accept
ct mark $uplink2_mark counter accept
#mark packets and connections with uplink addresses
ip saddr $uplink1_local_ip meta mark set $uplink1_mark ct mark set $uplink1_mark counter accept
ip saddr $uplink2_local_ip meta mark set $uplink2_mark ct mark set $uplink2_mark counter accept
#use load balancing for traffic from ```mail``` user
ip saddr != $uplink_local_ips skuid mail meta mark set $multipath_mark ct mark set $multipath_mark counter accept
#use load balancing for udp traffic
ip saddr != $uplink_local_ips udp dport 53 meta mark set $multipath_mark ct mark set $multipath_mark counter accept
}
}
table filter {
chain input {
type filter hook input priority 0
policy accept
#if you using accept policy, you can just accept marked packets here to avoid further processing and save a bit of cpu )
#mark packets/connections going through uplink devices
iifname $uplink1_dev meta mark set $uplink1_mark ct mark set $uplink1_mark counter
iifname $uplink2_dev meta mark set $uplink2_mark ct mark set $uplink2_mark counter
}
chain forward {
type filter hook forward priority 0
policy accept
tcp flags & (syn|rst) == syn tcp option maxseg size set rt mtu
#this is already marked connection already associated with some uplink, let packet go, do not need any handling
ct mark $uplink1_mark counter accept
ct mark $uplink2_mark counter accept
#mark packets/connections going through uplink devices
ip saddr $uplink1_local_ip meta mark set $uplink1_mark ct mark set $uplink1_mark counter accept
ip saddr $uplink2_local_ip meta mark set $uplink2_mark ct mark set $uplink2_mark counter accept
}
chain output {
type filter hook output priority 0
policy accept
#this is already marked connection already associated with some uplink, let packet go, do not need any handling
ct mark $uplink1_mark counter accept
ct mark $uplink2_mark counter accept
#mark packets/connections going from uplink addresses
ip saddr $uplink1_local_ip meta mark set $uplink1_mark ct mark set $uplink1_mark counter accept
ip saddr $uplink2_local_ip meta mark set $uplink2_mark ct mark set $uplink2_mark counter accept
}
}
little explanation:
#linux #vps #network #admin #sysadmin #iproute #iproute2 #iptables #netfilter #vpn #openvpn
some time ago i am faced interesting task, i needed to route some traffic via different uplinks (incoming and outgoing both). in linux it’s possible to use source based routing, uid/gid based routing via iproute2 and iptables.
/etc/iproute2/rt_tables
file already contain few predefined routing tables in following format:
<table_id> <table_name>
you can add something like:
1 my_table
you can ad as many table as you need.
ip route add default via 10.10.10.1 dev tap13 table my_table
ip route add 192.168.0.0/24 dev eth0 table my_table
something like this, second rule necessary for apps to be able to reach local network, if it’s needed.
ip rule add from 10.10.10.2/32 table my_table
this will route traffic from address 10.10.10.2 via “my_table” routing table
ip rule add fwmark 3 table my_table
this will route all traffic marked with “3” via “my_table” routing table, how to mark traffic will be explained later
iptables -t mangle -A OUTPUT -m owner --uid-owner pleroma -j MARK --set-mark 3
in step 3 i added rule to route all traffic marked with “3” via “my_table” routing table, and now i marked all traffic from uid pleroma with “3” so it will go via routing table as well. here required little hack for this to work, as by default “default” routing table is used, source address for this marked traffic most probably will be incorrect and routing will be confused and nothing will work, to prevent this we need to change source address like this:
iptables -t nat -A POSTROUTING -o tap13 -j SNAT --to-source=10.10.10.2
5.openvpn as you mat already noticed, i am using tap interface, it’s used by vpn implementation and as vpn implementation i using openvpn, i will not write much about openvpn here, instead i will post working openvpn configs for server and client, as it maybe a little tricky to write one:
server:
mode server
tls-server
port 12345
proto udp4
dev-type tap
dev tap13
ca /etc/openvpn/server/vps-ca.crt
cert /etc/openvpn/server/vps-server.crt
key /etc/openvpn/server/vps-server.key
dh /etc/openvpn/server/vps/dh4096.pem
server 10.10.10.0 255.255.255.0
push "route 10.10.10.0 255.255.255.0"
client-config-dir /etc/openvpn/server/vps/ccd
client-to-client
keepalive 30 240
max-clients 10
persist-key
persist-tun
status /var/log/openvpn/vps-status.log
verb 4
mute 20
tls-cipher "DHE-RSA-AES128-GCM-SHA256"
cipher AES-128-GCM
ncp-ciphers AES-128-GCM
tls-crypt /etc/openvpn/server/vps/shared_key
compress lzo
float
fast-io
verify-client-cert require
client:
tls-client
remote 127.0.0.1 #write server ip here instead
port 12345
proto udp4
dev-type tap
dev tap13
ca /etc/openvpn/client/vps-ca.crt
cert /etc/openvpn/client/vps-user.crt
key /etc/openvpn/client/vps-user.key
dh /etc/openvpn/dh4096.pem
keepalive 30 240
persist-key
persist-tun
verb 4
mute 20
tls-cipher "DHE-RSA-AES128-GCM-SHA256"
cipher AES-128-GCM
ncp-ciphers AES-128-GCM
tls-crypt /etc/openvpn/client/vps3-shared_key
compress lzo
float
fast-io
verify-client-cert require
aes cipher is used here because hw crypto available on cpu on both sides of connection, without hw crypto it’s better to set chacha20 for tls and cammelia for blocks instead of aes, 128 bit key size used for performance, if you need better security, set 256. most probably few options is useless in client config, like dh , compress, verify-client,keep-alive .
this short article does not cover firewall rules to route traffic from vps to client (this machine which we configuring in this article), it’s as trivial as FORWARD + PREROUTING, figure out it yourself as part of homework ) also, now exists another vpn implementations like wireguard https://www.wireguard.com/, which is kernel-space and should be more efficient than openvpn, also here is ipsec, l2tp and even pptp, i choose because it flexible and because i do not need very high bandwidth, so openvpn performance is suffice for me.