tested with nested 8.0.3e against 8.0.3d
this is only required when even the vcenter lost access to the esxi node. otherwise you can proceed with the easy route.
and a good amount of nerves and patience
you need to know what host network settings that esxi usually has – eventually deduce this information from another node
esxcli network ip interface ipv4 get
prepare the virtual medium e.g. on iDRAC
configuration > virtual media connect virtual media (upper right corner) map cd/dvd select file (click map device)
boot some system rescue system of your choice e.g. on iDRAC
F11 boot manager one-shot uefi boot menu virtual optical drive
boot your favorite rescue system e.g. sysrescuecd vs gentoo install/livecd
default boot entry, or incl. ram, or nomodeset if you insist gentoo live cd (cached if you wish, but it takes longer to start)
anyhow it’s nice to have good old nic names
e (edit) linux ... net.ifnames=0 ctrl-x (run)
assuming gentoo livecd
make the rescue system reachable by SSH somehow (here incl. VLAN tag)
rescue system console
for i in `seq 0 5`; do echo eth$i; ethtool eth$i | grep Link; done; unset i modprobe 8021q vconfig add eth2 VLAN_ID ifconfig eth2.VLAN_ID x.x.x.x/x ping -c3 x.x.x.x route add default gw x.x.x.x rc-service sshd start passwd
you can now reach the machine remotely
rescue system through ssh
ssh root@x.x.x.x fdisk -l mkdir lala/ mkdir lili/ mount /dev/nvme0n1p5 lala/ mount /dev/nvme0n1p5 lili/ ls -lh lala/*tgz ls -lh lili/*tgz md5sum lala/state.tgz md5sum lili/state.tgz
grab those and send them to the nested esxi
on workstation
scp root@x.x.x.x:/root/lala/state.tgz state5.tgz scp root@x.x.x.x:/root/lili/state.tgz state6.tgz scp state5.tgz nested-esxi:/var/tmp/ scp state6.tgz nested-esxi:/var/tmp/
the goal here is to enable real node encryption on the nested node – we do not change any content just yet (we simply recrypt the thing with, or at least enable, the target machine secret)
cd /var/tmp/ mkdir state5/ mkdir state6/ tar xzf state5.tgz -C state5/ tar xzf state6.tgz -C state6/ # no change there diff state5/encryption.info state6/encryption.info mkdir takeover/ cd takeover/ tar xzf /bootbank/state.tgz crypto-util envelope extract --aad ESXConfiguration local.tgz.ve local.tgz rm local.tgz.ve cp -f ../state5/encryption.info . tar czf takeover.tgz encryption.info local.tgz
not sure that’s enough (if not, proceed with the full-blown system rescue thing on the nested VM as well!)
cp -f takeover.tgz /bootbank/state.tgz cp -f takeover.tgz /altbootbank/state.tgz reboot
on the nested esxi
now that we have same encryption as on the real node, we can proceed with extracting the target image.
first, it’s a good time to compare what’s in the boot banks
cd /var/tmp/ cd state5/ crypto-util envelope extract --aad ESXConfiguration local.tgz.ve local.tgz tar xzf local.tgz rm -f local.tgz local.tgz.ve cd ../ cd state6/ crypto-util envelope extract --aad ESXConfiguration local.tgz.ve local.tgz tar xzf local.tgz rm -f local.tgz local.tgz.ve cd ../ diff -rbu state5/ state6/
on the nested esxi
grab some sha512 hash from another esxi node you know (worked here), or simply generate it as such (untested)
openssl passwd -6
fix the password hash
/usr/lib/vmware/sqlite/bin/sqlite3 \
var/lib/vmware/configstore/backup/current-store-1
select * from config where Name = 'user_accounts' and Identifier = 'root';
update config set UserValue='{"name":"root","password_hash":"YOUR-SHA512-HERE","description":"Administrator"}' where Name = 'user_accounts' and Identifier = 'root';
on the nested esxi
tar czf local.tgz .ssh/ etc/ var/ tar czf fixstate.tgz encryption.info local.tgz md5sum fixstate.tgz
on the workstation
scp nested-esxi:/var/tmp/state6/fixstate.tgz ./ scp fixstate.tgz real-esxi:
on the target machine
cp -f fixstate.tgz lala/state.tgz cp -f fixstate.tgz lili/state.tgz sync umount lala/ umount lili/ sync reboot
https://4sysops.com/archives/reset-forgotten-esxi-7x-8x-root-password-a-recovery-guide/
https://www.mwyann.us/posts/201 ==> very helpful! (but applies to esxi 7 not 8)
https://williamlam.com/2024/10/recovering-esxi-7-x-8-x-host-after-forgetting-or-losing-root-password.html ==> same trick and applies to esxi 8
https://pupuweb.com/how-to-reset-forgotten-esxi-7-x-or-8-x-root-password-without-reinstalling/
https://www.diskinternals.com/vmfs-recovery/vmfs-tools/
https://www.vinchin.com/tech-tips/mount-vmfs.html