AX86U crashes with IPSEC VPN accessed from guest

[gorstj]

AX86U Merlin 386.7_2 (with several AiMesh nodes via ethernet backhaul
PPPoE fibre broadband
5ghz guest is setup as per 'services-start' script below to run on br502 with all the vlan502 and wl1.1 (5gz guest) interfaces attached (there is an external DHCP server wired to that vlan and ports 500 & 4500 are punched through)
AX86U runs a IPSEC ikev2 vpn server via the GUI
Uninstalled most extensions/scripts (Skynet, Adguard etc as a trial)

The router is stable in the most part.

However, I can reliably cause a reboot by connecting to the guest LAN (either wl1.1 on the main router or coming from a AiMesh node on vlan502) then connecting to the IPSEC ikev2 VPN.
Connecting to the VPN via the main network (br0) or via the WAN works fine.

Looking at the crash log (attached) it looks like a process is causing a lockup - which process it is seems to be variable which doesn't help troubleshooting

Line 4942: May 5 06:05:08 crashlog: <0>NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [pdc_rx:270]
Line 7106: May 5 06:05:08 crashlog: <0>NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [httpd:2465]
Line 9301: May 5 06:05:08 crashlog: <0>NMI watchdog: BUG: soft lockup - CPU#3 stuck for 22s! [pdc_rx:270]
Line 12201: May 5 06:05:08 crashlog: <0>NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [pdc_rx:270]
Line 12202: May 5 06:05:08 crashlog: <0>NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [dropbear:22549]
Line 15536: May 5 06:05:08 crashlog: <0>NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [bcmsw_rx:259]

It is usually that the VPN stops responding for a few seconds then the direct connection stops working

This was consistent with a 'top' command I was performing just before one of todays reboots:

Mem: 517564K used, 416420K free, 2020K shrd, 7960K buff, 72684K cached
CPU0: 1.0% usr 3.0% sys 0.0% nic 41.8% idle 0.0% io 0.0% irq 54.0% sirq
CPU1: 0.0% usr 53.5% sys 0.0% nic 46.4% idle 0.0% io 0.0% irq 0.0% sirq
CPU2: 1.0% usr 3.0% sys 0.0% nic 96.0% idle 0.0% io 0.0% irq 0.0% sirq
CPU3: 0.0% usr 1.0% sys 0.0% nic 99.0% idle 0.0% io 0.0% irq 0.0% sirq
Load average: 1.20 1.22 1.01 7/198 21415
PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND
270 2 john RW 0 0.0 1 13.2 [pdc_rx]
259 2 john RW 0 0.0 0 13.0 [bcmsw_rx]
5393 3854 john S 731m 80.0 0 0.5 AdGuardHome -s run -c /opt/etc/AdGuardHome/AdGuardHome.yaml -w /opt/etc/AdGuardHome --pidfile /opt/var/run/AdGuardHome.pid --no-check-update -l syslog

services-start script

▒#remove VLAN 502 interfaces from br2
brctl delif br2 wl1.1
brctl delif br2 eth1.502
brctl delif br2 eth2.502
brctl delif br2 eth3.502
brctl delif br2 eth5.502
brctl delif br2 eth4.502
brctl delif br2 eth6.502
brctl delif br2 eth7.502

#add br502 interface and turn on STP
brctl addbr br502
brctl stp br502 on

#add VLAN502 interfaces to br502
brctl addif br502 wl1.1
brctl addif br502 eth1.502
brctl addif br502 eth2.502
brctl addif br502 eth3.502
brctl addif br502 eth4.502
brctl addif br502 eth5.502
brctl addif br502 eth6.502
brctl addif br502 eth7.502

#bring br502 up
ifconfig br502 up

#disable hardware switching (?cause of crashes without?) and restart EAPD (required?)
ethswctl -c hw-switching -o disable
killall eapd
eapd

#bring up br502 interface
ifconfig br502 [your IP] netmask 255.255.255.0
route add -net [network-range] netmask 255.0.0.0 gw [gateway-IP] br502

#open UDP ikev2 VPN to br502
iptables -I INPUT -i br502 -p udp -m multiport --dports 500,4500 -j ACCEPT

#not sure if this is necessary
nvram set lan2_ifnames="wl1.1 eth1.502 eth2.502 eth3.502 eth4.502 eth5.502 eth6.502 eth7.502"
nvram set lan2_ifname="br502"
nvram set br2_ifname="br502"
nvram commit

 

[gorstj]

Tried leaving all the Vlan502 interfaces on br502 (and removing the asus DHCP server config from this bridge)
Again, when connecting to VPN it causes a reboot after a short time period.

Now tried with an OpenVpn conenction instead..... works fine!
How odd - I will just use OpenVPN instead.