netbird | pfsync | haproxy | nginx | client
tested on debian12
┌──────────────────┐ CARP ┌──────────────────┐
│ │ PUBLIC VIP │ │
│ pfsync1 │ │ pfsync2 │
│ 10.1.0.28/16 │ CARP │ 10.1.0.29/16 │
│ │ INTERNAL VIP │ │
└──────────────────┘ 10.1.255.254/16 └──────────────────┘
│ │ │
│ │ │
│ │ └───────────────────┐
┌───────────────────────────────┘ │ │
│ │ 10000/tcp (http) │
80,443/tcp │ │ │
▼ │ 3478,5349/tcp │
┌────────────────────────┐ │ 3478,5349/udp │
│ │ │ 49152:49999/udp │
│ dnc-haproxy │ ▼ │
│ 10.1.0.27/16 │ ┌────────────────────────┐ │
│ ├────────────────► │ │ │
└────────────────────────┘ │ dnc-netbird │ ┌───────────┤
│ 10.1.0.20/16 │ │ │
│ │ │ 51853/udp │ 51854/udp
└────────────────────────┘ │ │
▼ ▼
┌──────────────┐ ┌──────────────┐
│ │ │ │
│ bastion53 │ │ bastion54 │
│ 10.1.0.53/16 │ │ 10.1.0.54/16 │
│ │ │ │
└──────────────┘ └──────────────┘
that poc needs to be on the public network as NETBIRD_DOMAIN resolves while SSL is required
– not sure it’s possible to PoC within with static name resolution
in the compose file, api mgmt endpoints need to be fixed manually
NETBIRD_MGMT_API_ENDPOINT NETBIRD_MGMT_GRPC_API_ENDPOINT
mgmt grpc needs to be reverse-proxied in a special way
as for signal grpc, we did not manage: haproxy was constantly reporting a 503, therefore we switched to plain text http for that haproxy backend.
you need an auth server and it needs to resolve from within the docker instances e.g.
ping -c1 keycloak.nethence.com
nmap -p 443 keycloak.nethence.com
see netbird-pfsync if you want DNAT
and eventually a reverse-proxy e.g. haproxy or nginx
unless you want docker-ce
apt install docker.io docker-compose docker ps apt install jq curl gettext-base
see keycloak and setup as such
realm netbird
user netbird & credentials (not temporary)
clients
client id netbird-client -- NETBIRD_AUTH_CLIENT_ID
+ oauth2 device auth grant
+ direct access grants
root url https://netbird.nethence.com/
redirect https://netbird.nethence.com/* & http://localhost:53000
post logout https://netbird.nethence.com/*
web origins "+"
client scopes "api" (default & OIDC)
+ include in token scope
new mapper type audience netbird-audience (client audience: netbird-client)
clients
netbird-client / client scopes --> api (default)
clients
client id netbird-backend -- NETBIRD_IDP_MGMT_CLIENT_ID
+ client auth
+ oauth2 device auth grant
+ direct access grants
+ service accounts roles
credentials tab / check credentials client secret -- NETBIRD_IDP_MGMT_CLIENT_SECRET
service accounts roles tab / assign view-users (realm mgmt)
grab latest tag
ver=0.41.0 cd /opt/ wget https://github.com/netbirdio/netbird/archive/refs/tags/v$ver.tar.gz tar xzf v$ver.tar.gz rm -f v$ver.tar.gz ln -s netbird-$ver netbird
cd netbird/infrastructure_files/ ls -lF setup.env.example curl ifconfig.me; echo # new file vi setup.env
NETBIRD_DOMAIN="netbird.nethence.com" NETBIRD_TURN_EXTERNAL_IP="@@@@@ front-facing public ip here @@@@@" # keycloak setup NETBIRD_AUTH_OIDC_CONFIGURATION_ENDPOINT="https://keycloak.nethence.com/realms/netbird/.well-known/openid-configuration" NETBIRD_USE_AUTH0=false NETBIRD_AUTH_CLIENT_ID="netbird-client" NETBIRD_AUTH_SUPPORTED_SCOPES="openid profile email offline_access api" NETBIRD_AUTH_AUDIENCE="netbird-client" NETBIRD_AUTH_DEVICE_AUTH_CLIENT_ID="netbird-client" NETBIRD_MGMT_IDP="keycloak" NETBIRD_IDP_MGMT_CLIENT_ID="netbird-backend" NETBIRD_IDP_MGMT_CLIENT_SECRET="@@@@@ netbird-backend client secret here @@@@@" NETBIRD_IDP_MGMT_EXTRA_ADMIN_ENDPOINT="https://keycloak.nethence.com/admin/realms/netbird" # we already got an ssl certificate NETBIRD_DISABLE_LETSENCRYPT=true NETBIRD_DISABLE_ANONYMOUS_METRICS=true NETBIRD_MGMT_DNS_DOMAIN=netbird.selfhosted #NETBIRD_MGMT_API_PORT= #NETBIRD_SIGNAL_PORT= #TURN_MIN_PORT="49152" #TURN_MAX_PORT="49999" NETBIRD_MGMT_DNS_DOMAIN=nethence.com NETBIRD_SIGNAL_PROTOCOL=http
proceed
cp -pi base.setup.env base.setup.env.dist
vi base.setup.env
# default https port is good enough
NETBIRD_MGMT_API_ENDPOINT=https://$NETBIRD_DOMAIN
cp -pi docker-compose.yml.tmpl docker-compose.yml.tmpl.dist
vi docker-compose.yml.tmpl
(make the dashboard listen 80 only)
s/:443//
(remove nginx ssl & letsencrypt volume)
dashboard:
signal:
relay:
management:
coturn:
extra_hosts:
- "keycloak.nethence.com=127.0.0.1"
- "netbird.nethence.com=127.0.0.1"
./configure.sh
cd artifacts/
check that you obtained the right kc setup (mgmt needs that to get a token endpoint)
#curl -s https://keycloak.nethence.com/realms/netbird/.well-known/openid-configuration > openid-configuration.json jq < openid-configuration.json vi docker-compose.yml
in our case we also had to add static name resolution (anyhow it might help, makes the path shorter)
vi management.json
"Signal": {
"Proto": "http",
#docker compose down docker compose up -d # --force-recreate docker ps -a netstat -lntup https://netbird.nethence.com/ ==> login as netbird (owner)
docker compose logs signal docker compose logs management docker compose logs coturn docker compose logs dashboard
Fatal glibc error: CPU does not support x86-64-v2
==> either rebuild keycloak (w/o RHEL9 as base) or change the VPS – works fine on xen/pvh
Error: failed retrieving a new idp manager with err: keycloak IdP configuration is incomplete, TokenEndpoint is missing
==> fix openid-configuration.json
https://docs.netbird.io/selfhosted/selfhosted-guide
https://docs.netbird.io/selfhosted/identity-providers#keycloak
https://docs.netbird.io/selfhosted/identity-providers#keycloak
https://github.com/netbirdio/netbird/issues/1247 ==> using advanced install and kc instead