GT-AX6000 reboots due to backblaze?

[Dragon]

Started a big backblaze refresh last night ( aka, reupload everything to shake old history ), and since then, router has been rebooting approx every 2 hours or so, with no messages in system log before reboot.

I assume it's crashing or running out of memory or something, but don't know how to try and pinpoint.

Any ideas? Router is usually reasonable stable, occasional random reboot, but only every few weeks or something to knowledge.

 

[dave14305]

with no messages in system log before reboot.

Are there any “crashlog” messages AFTER the reboot?

 

[Dragon]

In the regular system log? Not that I can see at a quick search, only mentions I see apart from partition creation:

May 5 08:05:16 kernel: 0x00000fe00000-0x00000fe80000 : "crashlog"

and

May 5 08:05:16 kernel: Kernel command line: coherent_pool=4M cpuidle_sysfs_switch pci=pcie_bus_safe console=ttyAMA0,115200 earlycon rootwait mtdparts=brcmnand.0:2097152(loader),264241152@2097152(image),524288@266338304(crashlog) root=/dev/ubiblock0_6 ubi.mtd=image ubi.block=0,6 rootfstype=squashfs cma=0M


I also note in logs that it seems to be reinstalling firmware each time currently? I did update it to latest .6 finally earlier today to see if this improved matter, but it didn't.

Edit: I see the firmware upload is actually the switch firmware being loaded.

 

[Dragon]

Just to add, seems to have stabilised a little more now that backblaze finished the bulk small file transfer and has switched to dealing with larger files, so it was likely something related to thousands of threads being opened and closed very quickly.

Edit: lasted another half hour or something before next reboot.

 

[sfx2000]

Just to add, seems to have stabilised a little more now that backblaze finished the bulk small file transfer and has switched to dealing with larger files, so it was likely something related to thousands of threads being opened and closed very quickly.

Yeah, you're likely running the NAT tables out of memory with 1000's of threads...

Not much different than BitTorrent when configured aggressively...

 

[AntonK]

Yeah, you're likely running the NAT tables out of memory with 1000's of threads...

Not much different than BitTorrent when configured aggressively...

Since I'm not a network tech I'll assume I'm missing something here, but the Backblaze settings allow you to configure the number of backup threads. I don't see how thousands of threads would apply here.

 

[Dragon]

I didn't mean simultanous threads, just thousands in a short period, I believe it runs a thread per file and does not keep it open between files, I've got the max simultanous set at 30. Observing the connections count in tools on router it gradually approximate grew in fluctuations up and down from 600 upto a 1000 or so, before dropping again, during that initial period. Currently it it seems to be hovering at around 90 active with ~450 tracked, still rebooting every now and then.

 

[Dragon]

These have continued with slightly lower average frequency on large file transfers, every 4-5 hours perhaps now, still no errors in log I can see.

 

[Dragon]

Would still like to try and figure out what the instability could have been, since the backup completed, normal service has again resumed and current uptime is over 7 days.