Any way to reboot a remote router when reboot is not rebooting it?

rjbu

Occasional Visitor
I have an RT-AC66U_B1 (384.15) router in a remote location - i.e. no one is there to power cycle it.
My devices in that location can connect to the router, but they can't get out to the internet anymore.
I cannot connect to the router thru https anymore (even if I could I doubt the reboot button would work)
but I can still ssh in and run commands.
I've tried scmerlin, and tell it to reboot the router; I've run "service reboot" (which is what scmerlin does);
I've tried "/sbin/reboot" and "/sbin/reboot -f" and nothing works.
If I look at what is in /tmp/syslog.log when I run /sbin/reboot -f I see
Mar 19 21:18:07 kernel: SysRq : Emergency Sync
but it doesn't reboot and stays up
Is there any other command to reboot the router?

In case anyone is wondering how the router gets in this state, I've seen this issue:
- The GUI System status screen shows 100% on Core 2
- top shows that it is not actual CPU usage on Core 2 (aka CPU1) but IO Wait
Code:
Mem: 211672K used, 44004K free, 908K shrd, 432K buff, 6620K cached
CPU0:  0.3% usr  0.0% sys  0.0% nic 97.0% idle  0.0% io  0.0% irq  2.5% sirq
CPU1:  0.7% usr  0.9% sys  0.0% nic  0.0% idle 97.4% io  0.0% irq  0.7% sirq
Load average: 1003.14 927.47 290.28 1/1185 23074
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
21424 21423 bob      R     1504  0.5   1  0.7 top
23004 23003 bob      S     1476  0.5   0  0.1 {scmerlin} /bin/sh /opt/bin/scmerl
23002   200 bob      S     1152  0.4   0  0.1 dropbear -p 2356 -s -a
  263     1 bob      D    23860  9.3   1  0.0 networkmap --bootwait
21431     1 bob      S     6124  2.3   1  0.0 httpd -i br0
 1278     1 bob      S     6000  2.3   1  0.0 /usr/sbin/smbd -D -s /etc/smb.conf
 1277     1 bob      S     5796  2.2   0  0.0 /usr/sbin/nmbd -D -s /etc/smb.conf
  393     1 bob      S     5640  2.2   1  0.0 amas_lib
  644   393 bob      S     5640  2.2   0  0.0 amas_lib
    1     0 bob      S     5624  2.2   1  0.0 /sbin/preinit
  194   187 bob      S     5624  2.2   0  0.0 /sbin/netool
  187     1 bob      S     5624  2.2   0  0.0 /sbin/netool
  195   194 bob      S     5624  2.2   0  0.0 /sbin/netool
  178     1 bob      S     5608  2.1   0  0.0 /sbin/wanduck
12816     1 bob      S     5604  2.1   0  0.0 bwdpi_wred_alive
  271     1 bob      S     5604  2.1   1  0.0 pctime
22926     1 bob      S     5604  2.1   0  0.0 watchdog
  270     1 bob      S     5604  2.1   0  0.0 hour_monitor
21308     1 bob      S     5604  2.1   0  0.0 usbled

- ps shows that a large number of processes show "D" state (I assume that is Disk Wait)
(I think whatever process gets in this state is just an innocent victim, not the cause)
- In the current case I see 1040 of these
Code:
  302 bob      1424 D    cp /tmp/syslog.log /tmp/syslog.log-1 /jffs
and now a number of other processes in "D" state
Code:
ps | grep D | less

    1 bob      5624 D    /sbin/preinit
   11 bob          0 DW  [sync_supers]
   36 bob          0 DW  [kworker/1:1]
   97 bob          0 DWN  [jffs2_gcd_mtd4]
  205 bob      5092 D    nt_center
  263 bob      23860 D    networkmap --bootwait

I've seen this now on 3 different RT-AC66U_B1 routers with 3 different software versions and 3 different usb sticks with 2 different file systems (once NTFS, twice ext4).
It's happens after the router has been up ~ 27 days.
 

L&LD

Part of the Furniture
Are you able to 'Safely remove' the USB drive? A reboot may be possible then?
 

dosborne

Very Senior Member
Not sure if it would help, but could you "kill" the seemingly runaway processes?
 

L&LD

Part of the Furniture
@JSinFCVA the USB doesn't need to be removed. :)

The ram and the processes just need to be 'released'. ;)
 

rjbu

Occasional Visitor
Not sure if it would help, but could you "kill" the seemingly runaway processes?
Sorry I didn't mention that I had tried that, but unfortunately the processes don't die and go away with "kill" or "kill -9"; think it's because they are in the "D" state waiting for disk io.
 

rjbu

Occasional Visitor
Are you able to 'Safely remove' the USB drive? A reboot may be possible then?
I don't have access to the GUI to do that anymore. Is there a command that the GUI runs to do that?
I'd be interested in trying that to see if it helps, but in the command that was blocked "cp /tmp/syslog.log /tmp/syslog.log-1 /jffs"
I didn't see anything that would be using the USB drive.
 

JDB

Very Senior Member
I don't have access to the GUI to do that anymore. Is there a command that the GUI runs to do that?
I'd be interested in trying that to see if it helps, but in the command that was blocked "cp /tmp/syslog.log /tmp/syslog.log-1 /jffs"
I didn't see anything that would be using the USB drive.

You could try umount /tmp/mnt/<usbname>

Add -l and or -f options to help force it.


Sent from my iPhone using Tapatalk
 

Dabombber

Senior Member
You might need to rearrange the order of things in your unmount script. Looks like you've still got stuff running which references /opt, so entware and/or your swap file might not have unmounted properly. If "cat /proc/swaps" shows anything you could try "swapoff -a".
 

CaptainSTX

Part of the Furniture
As another approach there are devices made for either automatically or from a remote location to power cycle network equipment in the case of failure, or using a schedule.

I happen to use an IPswitch which I got from 3GStore which continually pings five destinations and upon a certain number of failed pings to the five destinations power cycles both my modem and router until a connection is reestablished. It can also be set to power cycle the devices connected to its outlets. Finally it can be accessed remotely if you want force a reboot.
 

rjbu

Occasional Visitor
As another approach there are devices made for either automatically or from a remote location to power cycle network equipment in the case of failure, or using a schedule.

I happen to use an IPswitch which I got from 3GStore which continually pings five destinations and upon a certain number of failed pings to the five destinations power cycles both my modem and router until a connection is reestablished. It can also be set to power cycle the devices connected to its outlets. Finally it can be accessed remotely if you want force a reboot.
Good idea, thanks. I used to have a similar device that I would use to reboot my HTPC when I couldn't access it remotely.
 

rjbu

Occasional Visitor
I have an RT-AC66U_B1 (384.15) router in a remote location - i.e. no one is there to power cycle it.
My devices in that location can connect to the router, but they can't get out to the internet anymore.
I cannot connect to the router thru https anymore (even if I could I doubt the reboot button would work)
but I can still ssh in and run commands.
I've tried scmerlin, and tell it to reboot the router; I've run "service reboot" (which is what scmerlin does);
I've tried "/sbin/reboot" and "/sbin/reboot -f" and nothing works.
If I look at what is in /tmp/syslog.log when I run /sbin/reboot -f I see
Mar 19 21:18:07 kernel: SysRq : Emergency Sync
but it doesn't reboot and stays up
Is there any other command to reboot the router?
I finally noticed something in my original post that allowed me to answer my own question - SysRq
https://en.wikipedia.org/wiki/Magic_SysRq_key
specifically I was able to ssh in and reboot the router by running
Code:
echo b > /proc/sysrq-trigger
 

Yota

Very Senior Member
As another approach there are devices made for either automatically or from a remote location to power cycle network equipment in the case of failure, or using a schedule.

I happen to use an IPswitch which I got from 3GStore which continually pings five destinations and upon a certain number of failed pings to the five destinations power cycles both my modem and router until a connection is reestablished. It can also be set to power cycle the devices connected to its outlets. Finally it can be accessed remotely if you want force a reboot.
Just remember, that was last year, I was traveling in another country, and I used the router's openvpn server to protect my internet. when I left house, I added a smart plug to the router. so I can use my phone to control power on and off, and I think it will be good to restart the router.
The third day I was outside, I needed to restart my router. so I turned off the plug by app, and the plug and router never came online again. only then did I remember that the plug needed wifi.:oops:

I should use the routines function of the plug.:p
 
Last edited:

CaptainSTX

Part of the Furniture
If necessary I can log into my account with my ISP and trigger a router reboot. This will in turn cause my switch to reboot both my modem and my router.

Also the switch I mentioned has an Ethernet connection to the router eliminates the possibility of WiFi connectivity issues.

The only time I have been locked out was when a GFI breaker tripped. When I got back home I changed the breaker out.
 

Similar threads

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top