the re-learning NAT situation

IMAGE HERE

description

for the purpose of figuring out some tricks for the fuck martinez project part 4, I need to thoroughly check what’s possible and what is not in terms of NAT and static routing. Sorry if I’ve been a bit noisy on #networking@Libera, as it was a bit of a painful experimentation process.

the goal here is to let a connection pass through the lbs device with DNAT and then come back through the gw device through a totally seperated SNAT.

that could possibly be done with a VIP and synced conntrack states, but that would be an entirely other topic. also, using a VIP wouldn’t fit the purpose of splitting DNAT and SNAT apart.

warning

this architecture is NOT supposed to work (neither for TCP nor for UDP, as dst and src IPs are not the same) – I just wanted to break things and see what happens.

lessons learned

SNAT won’t trigger for TCP return path – probably because it’s not a SYN packet – while SNAT does trigger for UDP return datagrams (as there is no way to differenciate the datagram from being a query or an answer).

not only TCP won’t allow a return path to be different from the request path – UDP also isn’t happy with it, as the client says “port unreachable” tru ICMP when gw answers instead of lbs.

setup

the lbs setup (way in)

vi /etc/network/interfaces

auto eth0
iface eth0 inet static
    address 192.168.122.12/24
    gateway 192.168.122.1

auto eth1
iface eth1 inet static
    address 10.1.1.250/24

notice we do not even let connection tracking happen – as it would be useless anyhow in our case – therefore this becomes stateless NAT

vi /etc/nftables.conf

table ip nat
flush table ip nat
table ip nat {
    # DNAT
    chain prerouting {
        type nat hook prerouting priority dstnat;
        iif eth0 tcp dport 1234 notrack dnat 10.1.1.1;
        iif eth0 udp dport 1234 notrack dnat 10.1.1.1;
    }
}

the gw setup (way out)

vi /etc/network/interfaces

auto eth0
iface eth0 inet static
    address 192.168.122.11/24
    gateway 192.168.122.1

auto eth1
iface eth1 inet static
    address 10.1.1.254/24

stateless NAT here as well

vi /etc/nftables.conf

table ip nat
flush table ip nat
table ip nat {
    # SNAT
    chain postrouting {
        type nat hook postrouting priority srcnat;
        ip saddr 10.1.1.0/24 oif eth0 notrack snat 192.168.122.11;
    }
}

the server backend

notice default gateway points to the gw device, not the lbs device

vi /etc/network/interfaces

auto eth0
iface eth0 inet static
    address 10.1.1.1/24
    gateway 10.1.1.254

the client

the workstation is running GNS3 network emulator using KVM hence the default 192.168.122.0/24 subnet.

acceptance

while sniffing on the workstation

sudo tcpdump -ni virbr0 not arp and not tcp port 22

tcp

on server

    ncat -lvp 1234 -e /bin/bash

on workstation

    ncat -nv 192.168.122.12 1234

udp

on server

    ncat -ulvp 1234 -e /bin/bash

on workstation

    ncat -nvu 192.168.122.12 1234

results

with TCP, this is what I get – you can see that 10.1.1.1 does not get translated (although those packets went through the gw device) – this ends up in a timeout by itself, without even transmitting anything

16:13:46.623472 IP 192.168.122.1.35270 > 192.168.122.12.1234: Flags [S], seq 3507521535, win 64240, options [mss 1460,sackOK,TS val 11975077 ecr 0,nop,wscale 7], length 0
16:13:46.626234 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946328372 ecr 11975077,nop,wscale 5], length 0
16:13:47.628468 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946329374 ecr 11975077,nop,wscale 5], length 0
16:13:47.638629 IP 192.168.122.1.35270 > 192.168.122.12.1234: Flags [S], seq 3507521535, win 64240, options [mss 1460,sackOK,TS val 11976093 ecr 0,nop,wscale 7], length 0
16:13:47.641381 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946329387 ecr 11975077,nop,wscale 5], length 0
16:13:49.644204 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946331390 ecr 11975077,nop,wscale 5], length 0
16:13:49.654652 IP 192.168.122.1.35270 > 192.168.122.12.1234: Flags [S], seq 3507521535, win 64240, options [mss 1460,sackOK,TS val 11978109 ecr 0,nop,wscale 7], length 0
16:13:49.657468 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946331403 ecr 11975077,nop,wscale 5], length 0
16:13:53.676697 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946335422 ecr 11975077,nop,wscale 5], length 0
16:13:53.846632 IP 192.168.122.1.35270 > 192.168.122.12.1234: Flags [S], seq 3507521535, win 64240, options [mss 1460,sackOK,TS val 11982301 ecr 0,nop,wscale 7], length 0
16:13:53.849300 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946335594 ecr 11975077,nop,wscale 5], length 0
16:14:01.869836 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946343614 ecr 11975077,nop,wscale 5], length 0
16:14:17.999820 IP 10.1.1.1.1234 > 192.168.122.1.35270: Flags [S.], seq 822174118, ack 3507521536, win 65160, options [mss 1460,sackOK,TS val 1946359742 ecr 11975077,nop,wscale 5], length 0

with UDP, this is what I get

15:50:16.026928 IP 192.168.122.1.54911 > 192.168.122.12.1234: UDP, length 8
15:50:16.030093 IP 192.168.122.11.1234 > 192.168.122.1.54911: UDP, length 3
15:50:16.030163 IP 192.168.122.1 > 192.168.122.11: ICMP 192.168.122.1 udp port 54911 unreachable, length 39

additional notes

and yes I do have net.netfilter.nf_conntrack_tcp_loose enabled by default, that doesn’t help.