RT-AC1900 One Core 100% Usage

[Taddeusz]

I'm on the latest non-beta legacy firmware, 380.69. Every couple weeks I have an issue where I'll login to check things out and I'll notice that the first core has 100% load and the second has barely any. I'll try and reboot via the web interface and it will freeze up. To get it back up and running I must physically power it off and on. I saw another post that mentioned something about the QoS engine possibly causing issues like this. Any idea why this happens?

 

[RMerlin]

When that happens, connect over SSH and run "top". That will tell you what is using the CPU.

 

[FreshJR]

Also press "q" to quit top instead of closing the SSH terminal when you are done.

It will not auto exit by itself.

 

[Taddeusz]

I'll check the next time it happens and post back. I have not thought of doing that before. I tried rebooting it from work before lunch and it's currently in an unresponsive state until I get home this evening.

 

[Taddeusz]

Ok, I'm now on the latest 384.4_2 firmware and I've had this happen again. I'm trying to ssh into my router but all I get is "ASUSWRT-Merlin RT-AC68U 384.4-2 Sat Mar 24 17:01:45 UTC 2018" and no prompt ever appears so I can't get a top listing.

 

[Taddeusz]

Looks like you deleted your post. In the web interface core 1 varies a little but stays less than 5% most of the time while core 2 is a constant 100%. RAM usage is around 64%.

 

[Taddeusz]

Ok, getting this again after rebooting a couple days ago. On a previous snapshot I saw io at about 50%. Now its showing on nic.

PU:  0.0% usr  4.5% sys  0.0% nic 50.0% idle 45.4% io  0.0% irq  0.0% sirq
Load average: 7.00 7.09 7.13 1/117 3728
  PID  PPID USER     STAT   VSZ %VSZ CPU %CPU COMMAND
 3728  3560 taddeusz R     1416  0.5   0  4.5 top -n 1
  315     1 taddeusz S     6156  2.4   1  0.0 mastiff
 1965   334 taddeusz S     6156  2.4   0  0.0 mastiff
  334   315 taddeusz S     6156  2.4   0  0.0 mastiff
  335   334 taddeusz S     6156  2.4   0  0.0 mastiff
  383     1 taddeusz S     6072  2.3   1  0.0 /usr/sbin/smbd -D -s /etc/smb.conf
  381     1 taddeusz S     5908  2.3   0  0.0 nmbd -D -s /etc/smb.conf
  311     1 taddeusz D     5844  2.2   1  0.0 networkmap --bootwait
  283     1 taddeusz S     5652  2.2   1  0.0 httpds -s -i br0 -p 8443
  382   381 taddeusz S     5596  2.1   1  0.0 nmbd -D -s /etc/smb.conf
  288     1 taddeusz S     5372  2.1   0  0.0 watchdog
    1     0 taddeusz S     5320  2.0   0  0.0 /sbin/preinit
  246     1 taddeusz D     5308  2.0   1  0.0 nt_center
  226     1 taddeusz S     5308  2.0   1  0.0 /sbin/wanduck
  249   246 taddeusz S     5308  2.0   1  0.0 nt_center
  250   249 taddeusz S     5308  2.0   0  0.0 nt_center
  700     1 taddeusz S     5304  2.0   1  0.0 bwdpi_wred_alive
  385     1 taddeusz S     5304  2.0   0  0.0 erp_monitor
  454     1 taddeusz S     5304  2.0   0  0.0 ntp
  590     1 taddeusz S     5304  2.0   0  0.0 disk_monitor
  296   288 taddeusz S     5304  2.0   0  0.0 ots
  318     1 taddeusz S     5304  2.0   0  0.0 hour_monitor
  513     1 taddeusz S     5304  2.0   0  0.0 usbled
  254     1 taddeusz S     5304  2.0   0  0.0 wpsaide
  316     1 taddeusz S     5304  2.0   0  0.0 bwdpi_check
   91     1 taddeusz S     5300  2.0   1  0.0 console
  284     1 taddeusz S     5288  2.0   0  0.0 httpd -i br0
  266   243 taddeusz S     5152  2.0   0  0.0 nt_monitor
  244   243 taddeusz S     5152  2.0   1  0.0 nt_monitor
  231     1 taddeusz S     5152  2.0   0  0.0 nt_monitor
  243   231 taddeusz S     5152  2.0   1  0.0 nt_monitor
 3129     1 taddeusz S     4632  1.8   1  0.0 /etc/openvpn/vpnserver1 --cd /etc/openvpn/server1 --config config.ovpn
  588   587 taddeusz S     3812  1.4   0  0.0 cfg_server
  587   393 taddeusz S     3812  1.4   0  0.0 cfg_server
  393     1 taddeusz S     3812  1.4   0  0.0 cfg_server
11424   318 taddeusz D     3492  1.3   1  0.0 TrafficAnalyzer -e
  648   647 taddeusz S     2832  1.1   0  0.0 wred -B
  647   645 taddeusz S     2832  1.1   0  0.0 wred -B
  645     1 taddeusz S     2832  1.1   1  0.0 wred -B
  690   647 taddeusz S     2832  1.1   0  0.0 wred -B
  683   647 taddeusz S     2832  1.1   0  0.0 wred -B

 

[indark]

power cycle the router by power off the router for 30 sec to 5 min
hold the wps button while powering up, wait till the led start blinking then release the wps button for the nvram clear.


it solved similar issue for me couple firmware back, and someone else who I suggest to
https://www.snbforums.com/threads/r...4-384_20624-g14d2f02.45646/page-4#post-393382

hope it works for you as well.

 

[RMerlin]

Ok, getting this again after rebooting a couple days ago. On a previous snapshot I saw io at about 50%. Now its showing on nic.

The number is before the label, not after.

In this case, it's 50% idle (one core completely unused) and 45.4% IO (meaning either USB or network traffic most likely).

 

[Taddeusz]

The number is before the label, not after.

In this case, it's 50% idle (one core completely unused) and 45.4% IO (meaning either USB or network traffic most likely).

I don't have any USB devices hooked to it and there is barely any network traffic. Is there a way to figure out which IO is causing the load?

 

[Rob Q]

I had this happen to my RT-AC1900 just the other day. I wasn't sure how to fix it so I just pulled the power and plugged it back in again. Wonder why it does that. I'm running 384.4_2

 

[RMerlin]

I don't have any USB devices hooked to it and there is barely any network traffic. Is there a way to figure out which IO is causing the load?

Not that I'm aware of. This would be something fairly low-level within Linux.

 

[Rob Q]

Not that I'm aware of. This would be something fairly low-level within Linux.

Okay, I thought in my situation that it was doing something with the USB stick that I plugged in.
Could a script have bugged out and get stuck in a loop? Sometimes on my PC the internet browser bugs out on a webpage script and makes the CPU go on 100% load.

 

[RMerlin]

Okay, I thought in my situation that it was doing something with the USB stick that I plugged in.
Could a script have bugged out and get stuck in a loop? Sometimes on my PC the internet browser bugs out on a webpage script and makes the CPU go on 100% load.

If it were the case, then CPU usage would be tied to the process running the script rather than tied to IO.

 

[sfx2000]

f it were the case, then CPU usage would be tied to the process running the script rather than tied to IO

I wouldn't worry to much about it - top is kind of old, and single processor oriented... and the scripts do not take things into account.

TOP is about processes running, and processes ready to run, and this is averaged - with defaults, it's over 1 second, so it skews things a bit when one is considering how many events are happening over that second - 1GHz, do the math...

Science project that I had back last year - we would see TOP report 8 or above on a dual core ARMv7a (Armada), but dtrace showed that real-world utilization was much less than that, and the system was still responsive.

 

[Taddeusz]

I'm not sure this is related at all but many, if not most, times when I'm having this problem I go to reboot it through the web interface and it does nothing. I just looked in the log and it shows:

Apr  9 09:35:29 rc_service: httpds 283:notify_rc reboot
Apr  9 09:35:30 wan: [deconfig] udhcpc done[286]
Apr  9 09:35:30 kernel: Attempt to kill tasklet from interrupt
Apr  9 09:35:30 kernel: et0: et_mvlan_netdev_event: event 9 for vlan1 mvlan_en 0
Apr  9 09:35:30 kernel: et0: et_mvlan_netdev_event: event 2 for vlan1 mvlan_en 0
Apr  9 09:35:30 kernel: et0: et_mvlan_netdev_event: event 9 for vlan2 mvlan_en 0
Apr  9 09:35:30 miniupnpd[10491]: ioctl(s, SIOCGIFADDR, ...): Cannot assign requested address
Apr  9 09:35:30 miniupnpd[10491]: Failed to get IP for interface eth0
Apr  9 09:35:30 miniupnpd[10491]: SendNATPMPPublicAddressChangeNotification: cannot get public IP address, stopping
Apr  9 09:35:30 kernel: et0: et_mvlan_netdev_event: event 2 for vlan2 mvlan_en 0
Apr  9 09:35:30 kernel: et0: et_mvlan_netdev_event: event 13 for vlan1 mvlan_en 0
Apr  9 09:35:30 kernel: et0: et_mvlan_netdev_event: event 1 for vlan1 mvlan_en 0
Apr  9 09:35:30 kernel: et0: et_mvlan_netdev_event: event 13 for vlan2 mvlan_en 0
Apr  9 09:35:30 kernel: et0: et_mvlan_netdev_event: event 1 for vlan2 mvlan_en 0
Apr  9 09:35:30 ovpn-server1[3129]: Closing TUN/TAP interface
Apr  9 09:35:30 ovpn-server1[3129]: /usr/sbin/ip addr del dev tun21 10.8.0.1/24
Apr  9 09:35:30 ovpn-server1[3129]: SIGTERM[hard,] received, process exiting
Apr  9 09:35:32 iTunes: daemon is stopped
Apr  9 09:35:32 FTP_Server: daemon is stopped
Apr  9 09:35:33 Samba_Server: smb daemon is stopped
Apr  9 09:35:33 kernel: gro disabled
Apr  9 09:35:33 Timemachine: daemon is stopped
Apr  9 09:35:33 WEBDAV_Server: daemon is stopped
Apr  9 09:38:02 rc_service: httpds 283:notify_rc reboot
Apr  9 09:38:02 rc_service: waitting "reboot" via httpds ...
Apr  9 09:38:17 rc_service: skip the event: reboot.

Not sure why it skipping the reboot event.

 

[Taddeusz]

I even tried reboot from ssh and it does nothing.

Web interface is still working but internet connection now shows "disconnected". I can still connect to my services remotely.

 

[Taddeusz]

And now my home has gone unresponsive. Has been nearly 30 minutes since I initiated a reboot.

 

[RMerlin]

Power cycle the router. It got stuck on the first reboot request - subsequent requests were dropped because the first one has never completed.

 

[Taddeusz]

Power cycle the router. It got stuck on the first reboot request - subsequent requests were dropped because the first one has never completed.

I will when I get home this evening. Till then it's stuck.