Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

AC68U Core 1 or 2 goes 100% after a couple weeks

Discussion in 'Asuswrt-Merlin' started by HiVolt, Jan 17, 2017.

  1. HiVolt

    HiVolt Occasional Visitor

    Joined:
    Jan 9, 2014
    Messages:
    11
    I have a an AC68U running Merlin 380.63_2, I've already updated a couple times from minor versions before...

    Here's a problem I've got. After about 2 weeks runtime, one of the cores, sometimes its Core 1 sometimes Core 2 gets pegged at 100%. I telnet to the router, and run "top" and this is what it shows...

    Mem: 73136K used, 182564K free, 0K shrd, 2120K buff, 8276K cached
    CPU: 0.0% usr 0.1% sys 0.0% nic 47.3% idle 47.9% io 0.0% irq 4.5% sirq
    Load average: 4.03 4.04 4.05 1/85 13805

    Looks like NIC is showing nearly 50% usage so i assume whatever this is, its hogging the resources. None of the actual processes are showing high usage. The router still works fine I don't see any adverse effects but If i reboot it it will be fine for a couple weeks then it will happen again.

    I'm running it in Dual WAN mode if it makes a difference, with primary WAN port being Fibre 250/250, and a failover Cable 30/5 connection on LAN port 4.

    Any help would be appreciated, thanks.

    100_core1.JPG
     
  2. john9527

    john9527 Part of the Furniture

    Joined:
    Mar 28, 2014
    Messages:
    4,639
    Location:
    United States
    labels are after the value....so you are

    47.3% idle
    47.9% io

    Do you have a USB drive connected? Running Media Server? Also, check the syslog for any disk errors.
     
  3. HiVolt

    HiVolt Occasional Visitor

    Joined:
    Jan 9, 2014
    Messages:
    11
    Oh, sorry. I thought it was NIC...

    Nope, no media server, no usb connected... this is in an office purely as a router not running any fancy stuff...
     
  4. HiVolt

    HiVolt Occasional Visitor

    Joined:
    Jan 9, 2014
    Messages:
    11
    I went thru all the options, there are no AI protection, no AI cloud turned on, no VPN, no media servers. As I was going thru the web interface crashed... So i had to reboot...

    I would like to figure this out somehow, its gonna be a bit of a pain in the butt if it keeps happening.
     
  5. john9527

    john9527 Part of the Furniture

    Joined:
    Mar 28, 2014
    Messages:
    4,639
    Location:
    United States
    Do you have a copy of the top output? If not, next time it happens see what process is at the top of the 'top' list.
     
  6. Reinvented

    Reinvented Regular Contributor

    Joined:
    Aug 10, 2015
    Messages:
    96
    When you were flashing different firmwares, I'm assuming you were dirty flashing, and also loading up a config file as well? Or were you setting up stuff from scratch after clearing nvram?
     
  7. HiVolt

    HiVolt Occasional Visitor

    Joined:
    Jan 9, 2014
    Messages:
    11
    This is fairly new router only set up less than 2 months... When I first got it, I flashed it with latest Merlin and reset all values to factory... I have since flashed it twice with incremental updates hoping to fix this glitch, but did not clear or reset anything, nor did I load any config file.

    I'm sure this will happen again in ~2weeks, so I will post the "top" output then. But from what I recall there was no process with higher than 1-2% thruout the list. Thats what baffled me.
     
  8. RMerlin

    RMerlin Part of the Furniture

    Joined:
    Apr 14, 2012
    Messages:
    23,925
    Location:
    Canada
    Check your traffic meter while that happens. Generally, io usage will be either USB or network.
     
  9. HiVolt

    HiVolt Occasional Visitor

    Joined:
    Jan 9, 2014
    Messages:
    11
    I did, there was minimal traffic and there has never been any USB plugged in. This has happened 3 times alerady, and it's roughly 2 weeks after boot/reboot. If you have specific things I should check next time I'm willing to provide you with any details you need.

    I remember checking the traffic before when maxing out the line at 250mbps, and only one of the cores was around 40%, so one core being stuck at 100% shouldn't be a result of any network traffic IMO.
     
  10. HiVolt

    HiVolt Occasional Visitor

    Joined:
    Jan 9, 2014
    Messages:
    11
    I hate to reply to this old thread, but I'm still having this issue, on and off... It has once gone like 3-4 months without one of the cores being pegged at 100%, but sometimes it does it like a week after reboot...

    I have upgraded the firmware several times and currently sit at Merlin 380.68 on the AC68u. If you google this problem, it seems it happens to all kinds of ASUS routers on stock or Merlin fw... It's unfortunate this particular revision C does not support Tomato or I would try it to see if it screws up.

    The TOP readout is below.

    Mem: 74252K used, 181456K free, 1116K shrd, 1072K buff, 7664K cached
    CPU: 0.3% usr 0.2% sys 0.0% nic 46.4% idle 48.1% io 0.0% irq 4.7% sirq
    Load average: 5.04 5.08 5.11 1/91 10273
    PID PPID USER STAT VSZ %VSZ CPU %CPU COMMAND
    1554 1 admin S 916 0.3 0 0.4 miniupnpd -f /etc/upnp/config
    480 1 admin S 1320 0.5 1 0.2 protect_srv
    504 1 admin S 3092 1.2 1 0.1 avahi-daemon: running [RT-AC68U-DB38.local]
    511 1 admin S 7288 2.8 0 0.0 watchdog
    506 1 admin S 6708 2.6 0 0.0 httpd -i br0
    505 1 admin S 6364 2.4 0 0.0 httpds -s -p 8443 -i br0
    1 0 admin S 6232 2.4 0 0.0 /sbin/preinit
    954 1 admin S 6220 2.4 0 0.0 bwdpi_wred_alive
    479 1 admin S 6220 2.4 0 0.0 /sbin/wanduck
    531 1 admin S 6220 2.4 0 0.0 erp_monitor
    847 1 admin S 6220 2.4 0 0.0 ntp
    4271 1 admin S 6220 2.4 0 0.0 usbled
    529 1 admin S 6220 2.4 1 0.0 hour_monitor
    897 1 admin S 6220 2.4 1 0.0 disk_monitor
    515 511 admin S 6220 2.4 0 0.0 ots
    491 1 admin S 6220 2.4 1 0.0 wpsaide
    528 1 admin S 6220 2.4 1 0.0 bwdpi_check
    352 1 admin S 6212 2.4 0 0.0 console
    10850 10849 admin D 5252 2.0 0 0.0 bwdpi_sqlite -e -s NULL
    963 959 admin S 4004 1.5 1 0.0 data_colld -i 1800 -p 43200 -b -w /tmp/bwdpi/dc/
    958 1 admin S 4004 1.5 0 0.0 data_colld -i 1800 -p 43200 -b -w /tmp/bwdpi/dc/
    974 959 admin S 4004 1.5 0 0.0 data_colld -i 1800 -p 43200 -b -w /tmp/bwdpi/dc/
    959 958 admin S 4004 1.5 0 0.0 data_colld -i 1800 -p 43200 -b -w /tmp/bwdpi/dc/
    961 959 admin S 4004 1.5 0 0.0 data_colld -i 1800 -p 43200 -b -w /tmp/bwdpi/dc/
    962 959 admin S 4004 1.5 0 0.0 data_colld -i 1800 -p 43200 -b -w /tmp/bwdpi/dc/
    960 959 admin S 4004 1.5 0 0.0 data_colld -i 1800 -p 43200 -b -w /tmp/bwdpi/dc/
    915 914 admin S 2724 1.0 0 0.0 wred -B
    914 913 admin S 2724 1.0 1 0.0 wred -B
    913 1 admin S 2724 1.0 0 0.0 wred -B
    916 914 admin S 2724 1.0 0 0.0 wred -B
    917 914 admin S 2724 1.0 1 0.0 wred -B
    922 914 admin S 2724 1.0 1 0.0 wred -B
    923 914 admin S 2724 1.0 0 0.0 wred -B
    920 914 admin S 2724 1.0 1 0.0 wred -B
    921 914 admin S 2724 1.0 0 0.0 wred -B
    918 914 admin S 2724 1.0 1 0.0 wred -B
    919 914 admin S 2724 1.0 0 0.0 wred -B
    516 1 admin S 2296 0.9 1 0.0 rstats
    4276 4269 admin S 2200 0.8 1 0.0 u2ec
    4269 1 admin S 2200 0.8 0 0.0 u2ec
    4277 4276 admin S 2200 0.8 1 0.0 u2ec
    4266 1 admin D 1984 0.7 0 0.0 networkmap
    4237 1 admin S 1832 0.7 1 0.0 nas
    4238 1 admin S 1804 0.7 1 0.0 /bin/wps_monitor
    1557 1 admin S 1452 0.5 0 0.0 /sbin/udhcpc -i vlan3 -p /var/run/udhcpc1.pid -s /tmp/udhcpc -O33 -O249
    482 1 admin S 1452 0.5 0 0.0 telnetd -b 192.168.1.1
    10169 482 admin S 1448 0.5 1 0.0 -sh
    10246 10169 admin R 1448 0.5 0 0.0 top
    354 352 admin S 1444 0.5 0 0.0 /bin/sh
    10933 511 admin D 1444 0.5 0 0.0 cp /tmp/syslog.log /tmp/syslog.log-1 /jffs
    508 1 admin S 1440 0.5 1 0.0 crond
    369 1 admin S 1440 0.5 0 0.0 /sbin/klogd -c 5
    10849 529 admin S 1440 0.5 1 0.0 sh -c bwdpi_sqlite -e -s NULL
    367 1 admin S 1436 0.5 1 0.0 /sbin/syslogd -m 0 -S -O /tmp/syslog.log -s 256 -l 7
    550 1 admin S 1436 0.5 1 0.0 /sbin/zcip -p /var/run/zcip0.pid vlan35 /tmp/zcip
    553 1 admin S 1392 0.5 0 0.0 /usr/sbin/pppd file /tmp/ppp/options.wan0
    15686 1 nobody S 1376 0.5 0 0.0 dnsmasq --log-async
    486 480 admin S 1320 0.5 1 0.0 protect_srv
    487 486 admin S 1320 0.5 0 0.0 protect_srv
    4250 1 admin S 1316 0.5 1 0.0 /usr/sbin/acsd
    4270 1 admin S 1236 0.4 0 0.0 lpd br0
    523 1 admin S 1208 0.4 1 0.0 lld2d br0
    4235 1 admin S 1192 0.4 1 0.0 /bin/eapd
    509 1 admin S 1164 0.4 1 0.0 /usr/sbin/infosvr br0
    293 1 admin S 664 0.2 0 0.0 hotplug2 --persistent --no-coldplug
    [email protected]:/tmp/home/root#
     
  11. VZ3

    VZ3 Occasional Visitor

    Joined:
    Nov 4, 2016
    Messages:
    49
    I have the same issue with AC87R and 380.68. It something with disk IO, somewhere in kernel. There is nothing unusual to see in TOP command other that one CPU is waiting/looping/got stuck on IO.

    PS: I have an impression that Web UI is kinda facilitated/triggered that bug. Or maybe it got stuck when trying to write log or traffic db into jffs. So, after that traffic.db got corrupted, so I have to delete that file.
     
    Last edited: Sep 12, 2017
  12. selenajohn

    selenajohn New Around Here

    Joined:
    Sep 12, 2017
    Messages:
    6
    labels are after the value....so you are

    47.3% idle
    47.9% io
     
  13. john9527

    john9527 Part of the Furniture

    Joined:
    Mar 28, 2014
    Messages:
    4,639
    Location:
    United States
    Also, if you press the '1' key after starting top, it will show your the stats for each CPU individually.
     
  14. ColinTaylor

    ColinTaylor Part of the Furniture

    Joined:
    Mar 31, 2014
    Messages:
    3,508
    Location:
    UK
    As previously pointed out in post #2. :rolleyes:
     
  15. VZ3

    VZ3 Occasional Visitor

    Joined:
    Nov 4, 2016
    Messages:
    49
    Again, there is nothing to see in top command except that one core stuck with high IO.
     
  16. Vexira

    Vexira Very Senior Member

    Joined:
    Jan 20, 2017
    Messages:
    668
    Location:
    Australia
    enable reboot scheduler and set it to a time when there's no running traffic, say 4am that what I did. But mines set to reboot once a day to keep every thing running smoothly you might what it to reboot once a week.
     
  17. HiVolt

    HiVolt Occasional Visitor

    Joined:
    Jan 9, 2014
    Messages:
    11
    I thought about that, except I have mine set up in a Dual WAN config, Primary being FTTH with PPPoE authentication, and the Secondary backup being a Cable connection with DHCP, and after a reboot the secondary doesn't get its DHCP IP, i have to toggle it manually to static then back to DHCP and then it works...
     
  18. HiVolt

    HiVolt Occasional Visitor

    Joined:
    Jan 9, 2014
    Messages:
    11
    Mem: 72940K used, 182768K free, 1148K shrd, 732K buff, 7164K cached
    CPU0: 0.3% usr 0.9% sys 0.0% nic 96.8% idle 0.0% io 0.0% irq 1.7% sirq
    CPU1: 0.0% usr 0.5% sys 0.0% nic 0.0% idle 96.0% io 0.0% irq 3.3% sirq
    Load average: 5.00 5.02 5.05 1/91 9368

    That just shows me the same thing as the main GUI page. That's how I notice that one of the CPU's has gone 100%, I don't check via terminal.

    I just don't get why this problem has persisted for this long on these routers... There has to be some core flaw in the firmware, I don't think it would be hardware since there are many varieties.

    I don't have any USB disks plugged in, I don't use it as any sort of media server, I have nothing connecting in or out of it. I just use it as a router.
     
  19. john9527

    john9527 Part of the Furniture

    Joined:
    Mar 28, 2014
    Messages:
    4,639
    Location:
    United States
    Here's a SWAG.....do a backup of JFFS, then select 'Reformat at next boot', reboot the router twice in a row (sometimes it takes two passes for a successful reformat), then restore the JFFS backup and reboot again. Maybe there's a bad block in jffs that's not being handled correctly.
     
  20. HiVolt

    HiVolt Occasional Visitor

    Joined:
    Jan 9, 2014
    Messages:
    11
    Whats the JFFS partition do anyway?

    Format JFFS partition at next boot - NO
    Enable JFFS custom scripts and configs - NO

    Those are the settings I have. If there is no scripts/configs turned on, does that matter?
     

Share This Page