for the purpose of figuring out some tricks for the fuck martinez project part 4, I need to thoroughly check what’s possible and what is not in terms of NAT and static routing. Sorry if I’ve been a bit noisy on #networking@Libera, as it was a bit of a painful experimentation process.
the goal here is to let a connection pass through the lbs device with DNAT and then come back through the gw device through a totally seperated SNAT.
that could possibly be done with a VIP and synced conntrack states, but that would be an entirely other topic. also, using a VIP wouldn’t fit the purpose of splitting DNAT and SNAT apart.
this architecture is NOT supposed to work (at least with TCP and in some extend even apparently with UDP) – I just wanted to break things and see what happens.
SNAT won’t trigger for TCP return path – probably because it’s not a SYN packet – while SNAT does trigger for UDP return datagrams (as there is no way to differenciate the datagram from being a query or an answer).
not only TCP won’t allow a return path to be different from the request path – UDP also isn’t happy with it, as the client says “port unreachable” tru ICMP when gw answers instead of lbs.
vi /etc/network/interfaces
auto eth0
iface eth0 inet static
address 192.168.122.12/24
gateway 192.168.122.1
auto eth1
iface eth1 inet static
address 10.1.1.250/24
notice we do not even let connection tracking happen – as it would be useless anyhow in our case – therefore this becomes stateless NAT
vi /etc/nftables.conf
table ip nat
flush table ip nat
table ip nat {
# DNAT
chain prerouting {
type nat hook prerouting priority dstnat;
iif eth0 tcp dport 1234 notrack dnat 10.1.1.1;
iif eth0 udp dport 1234 notrack dnat 10.1.1.1;
}
}
vi /etc/network/interfaces
auto eth0
iface eth0 inet static
address 192.168.122.11/24
gateway 192.168.122.1
auto eth1
iface eth1 inet static
address 10.1.1.254/24
stateless NAT here as well
vi /etc/nftables.conf
table ip nat
flush table ip nat
table ip nat {
# SNAT
chain postrouting {
type nat hook postrouting priority srcnat;
ip saddr 10.1.1.0/24 oif eth0 notrack snat 192.168.122.11;
}
}
notice default gateway points to the gw device, not the lbs device
vi /etc/network/interfaces
auto eth0
iface eth0 inet static
address 10.1.1.1/24
gateway 10.1.1.254
the workstation is running GNS3 network emulator using KVM hence the default 192.168.122.0/24 subnet.
while sniffing on the workstation
sudo tcpdump -ni virbr0 not arp and not tcp port 22
on server
ncat -lvp 1234 -e /bin/bash
on workstation
ncat -nv 192.168.122.12 1234
on server
ncat -ulvp 1234 -e /bin/bash
on workstation
ncat -nvu 192.168.122.12 1234
with TCP, this is what I get – you can see that 10.1.1.1 does not get translated (although those packets went through the gw device) – this ends up in a timeout by itself, without even transmitting anything
16:13:46.623472 IP 192.168.122.1.35270 > 192.168.122.12.1234: Flags [S], seq 3507521535, win 64240, options [mss 1460,sackOK,TS val 11975077 ecr 0,nop,wscale 7], length 0 16:13:46.626234 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946328372 ecr 11975077,nop,wscale 5], length 0 16:13:47.628468 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946329374 ecr 11975077,nop,wscale 5], length 0 16:13:47.638629 IP 192.168.122.1.35270 > 192.168.122.12.1234: Flags [S], seq 3507521535, win 64240, options [mss 1460,sackOK,TS val 11976093 ecr 0,nop,wscale 7], length 0 16:13:47.641381 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946329387 ecr 11975077,nop,wscale 5], length 0 16:13:49.644204 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946331390 ecr 11975077,nop,wscale 5], length 0 16:13:49.654652 IP 192.168.122.1.35270 > 192.168.122.12.1234: Flags [S], seq 3507521535, win 64240, options [mss 1460,sackOK,TS val 11978109 ecr 0,nop,wscale 7], length 0 16:13:49.657468 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946331403 ecr 11975077,nop,wscale 5], length 0 16:13:53.676697 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946335422 ecr 11975077,nop,wscale 5], length 0 16:13:53.846632 IP 192.168.122.1.35270 > 192.168.122.12.1234: Flags [S], seq 3507521535, win 64240, options [mss 1460,sackOK,TS val 11982301 ecr 0,nop,wscale 7], length 0 16:13:53.849300 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946335594 ecr 11975077,nop,wscale 5], length 0 16:14:01.869836 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946343614 ecr 11975077,nop,wscale 5], length 0 16:14:17.999820 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946359742 ecr 11975077,nop,wscale 5], length 0
with UDP, this is what I get
15:50:16.026928 IP 192.168.122.1.54911 > 192.168.122.12.1234: UDP, length 8 15:50:16.030093 IP 192.168.122.11.1234 > 192.168.122.1.54911: UDP, length 3 15:50:16.030163 IP 192.168.122.1 > 192.168.122.11: ICMP 192.168.122.1 udp port 54911 unreachable, length 39
and yes I do have net.netfilter.nf_conntrack_tcp_loose enabled by default, that doesn’t help.
try sync conntrack states between gw and lbs and see what happens (IPs are different)