RT-AC66U w/asuswrt-merlin 378.55 stops fully booting

[Tobias McNulty]

I'm running into a strange issue where my router crashes and then won't fully boot up again, or at least that's what's happening as far as I can tell. Everything is operating normally, then without warning the router stops working. I try to reboot it, but it won't start back up again (even after being off for a long time) without reflashing the firmware. When it happened the first time I hoped it was just a one-time thing and didn't worry about it too much, but now it's happened again. I've noticed that if I give my computer a static IP and connect to the router via ethernet, I can SSH to the router, but that's about all. Wifi, DHCP, and the web interface all do NOT come up. The router is running 378.55 (17-July-2015).

I didn't reflash the router yet in case I can do any debugging that might help figure out what's going on. Has anyone seen this before, or does anyone have any suggestions for where to look first for debugging?

Many thanks,
Tobias

 

[L&LD]

First step in debugging. Flash the latest RMerlin firmware; 378.56_2.

No point in anyone wasting time on firmware that is now effectively obsolete.

In my experience, the RT-AC66U was not a very stable or dependable model (had more than a few that simply quit working for customers). You might try using a known good ac power adaptor, but it seems like a new router is your best move today.

 

[Tobias McNulty]

Sure hope I don't need a new router (this was supposed to be the new/stable/fast one), but what would you recommend instead? Also, I didn't see anything in the changelog related to this, which is why I hesitate to upgrade without at least first backing up the filesystem for later inspection (which sounds like a better plan, now that I think of it). Would very much like to get to the bottom of this...

Sent from my XT1095 using Tapatalk

 

[Tobias McNulty]

I was wrong...the router does not even enter recovery mode properly when it gets in this state. All I can do is ping the router on its configured IP (not 192.168.1.1); no SSH, no HTTP.

To fix the issue I booted the router in regular (non-recovery) mode, scp a new firmware image to /tmp, SSH to the router, and write the new image using the mtd-write command:

$ scp RT-AC66U_378.56_2.trx admin@10.30.2.1:/tmp
$ scp admin@10.30.2.1
# mtd-write -i /tmp/RT-AC66U_378.56_2.trx -p linux
# reboot

Now, the router reboots with all of my original settings intact, and the new firmware (!). How is this possible? I understand the user settings are stored on a separate partition that was not reflashed, but it's still bizarre to me that the firmware was corrupted (?), could not fully boot, but was still operational enough to reflash a new firmware image.

Here's the output of a few commands from the router *before* reflashing the firmware:

ps:

admin@rtr:/tmp/home/root# ps
  PID USER       VSZ STAT COMMAND
    1 admin     5020 R    /sbin/init
    2 admin        0 SW<  [kthreadd]
    3 admin        0 SWN  [ksoftirqd/0]
    4 admin        0 SW<  [events/0]
    5 admin        0 SW<  [khelper]
   20 admin        0 SW<  [kblockd/0]
   51 admin        0 SW   [pdflush]
   52 admin        0 SW   [pdflush]
   53 admin        0 SW<  [kswapd0]
   54 admin        0 SW<  [aio/0]
   98 admin        0 SW<  [mtdblockd]
  121 admin        0 SW<  [kmmcd]
  125 admin      648 S    hotplug2 --persistent --no-coldplug
  163 admin     5004 S    console
  167 admin     1432 S    /bin/sh
  172 admin        0 SWN  [jffs2_gcd_mtd5]
  175 admin     1420 S    syslogd -m 0 -S -O /tmp/syslog.log -s 256 -l 7 -R 127.0.0.1:514 -L
  177 admin     1420 S    /sbin/klogd
  268 admin     5012 S    /sbin/wanduck
  271 admin     1128 S    dropbear -p 22 -s -j -k
  272 admin     1176 R    dropbear -p 22 -s -j -k
  273 admin     1432 S    -sh
  277 admin     1424 R    ps


admin@rtr:/tmp/home/root# mount
rootfs on / type rootfs (rw)
/dev/root on / type squashfs (ro)
proc on /proc type proc (rw)
tmpfs on /tmp type tmpfs (rw)
devfs on /dev type tmpfs (rw,noatime)
sysfs on /sys type sysfs (rw)
devpts on /dev/pts type devpts (rw)
/dev/mtdblock5 on /jffs type jffs2 (rw,noatime)

admin@rtr:/tmp/home/root# dmesg
start_kernel
Linux version 2.6.22.19 (root@asus) (gcc version 4.2.3) #1 Thu Jul 16 23:18:18 EDT 2015
CPU revision is: 00019749
Found an ST compatible serial flash with 32 64KB blocks; total size 2MB
Determined physical RAM map:
memory: 07fff000 @ 00000000 (usable)
memory: 08000000 @ 87fff000 (usable)
On node 0 totalpages: 589823
  Normal zone: 1024 pages used for memmap
  Normal zone: 0 pages reserved
  Normal zone: 130048 pages, LIFO batch:31
  HighMem zone: 3583 pages used for memmap
  HighMem zone: 455168 pages, LIFO batch:31
Built 1 zonelists.  Total pages: 585216
Kernel command line: root=/dev/mtdblock3 console=ttyS0,115200 init=/sbin/preinit
Primary instruction cache 32kB, physically tagged, 4-way, linesize 32 bytes.
Primary data cache 32kB, 4-way, linesize 32 bytes.
Synthesized TLB refill handler (20 instructions).
Synthesized TLB load handler fastpath (32 instructions).
Synthesized TLB store handler fastpath (32 instructions).
Synthesized TLB modify handler fastpath (31 instructions).
PID hash table entries: 2048 (order: 11, 8192 bytes)
CPU: BCM5300 rev 1 pkg 0 at 600 MHz
Using 300.000 MHz high precision timer.
console [ttyS0] enabled
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Memory: 238352k/131068k available (2667k kernel code, 22808k reserved, 532k data, 200k init, 131072k highmem)
Calibrating delay loop... 299.82 BogoMIPS (lpj=1499136)
Mount-cache hash table entries: 512
NET: Registered protocol family 16
PCI: Initializing host
PCI: Reset RC
PCI: Initializing host
PCI: Reset RC
PCI: Fixing up bus 0
PCI/PCIe coreunit 0 is set to bus 1.
PCI: Fixing up bridge
PCI: Setting latency timer of device 0000:01:00.0 to 64
PCI: Fixing up bridge
PCI: Setting latency timer of device 0000:01:00.1 to 64
PCI: Enabling device 0000:01:00.1 (0004 -> 0006)
PCI: Fixing up bus 1
PCI/PCIe coreunit 1 is set to bus 2.
PCI: Fixing up bridge
PCI: Setting latency timer of device 0000:02:00.0 to 64
PCI: Fixing up bridge
PCI: Setting latency timer of device 0000:02:00.1 to 64
PCI: Enabling device 0000:02:00.1 (0004 -> 0006)
PCI: Fixing up bus 2
NET: Registered protocol family 2
Time: MIPS clocksource has been installed.
IP route cache hash table entries: 4096 (order: 2, 16384 bytes)
TCP established hash table entries: 16384 (order: 5, 131072 bytes)
TCP bind hash table entries: 16384 (order: 4, 65536 bytes)
TCP: Hash tables configured (established 16384 bind 16384)
TCP reno registered
highmem bounce pool size: 64 pages
squashfs: version 3.2-r2 (2007/01/15) Phillip Lougher
io scheduler noop registered (default)
HDLC line discipline: version $Revision: 4.8 $, maxframe=4096
N_HDLC line discipline registered.
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing disabled
serial8250: ttyS0 at MMIO 0x0 (irq = 8) is a 16550A
serial8250: ttyS1 at MMIO 0x0 (irq = 8) is a 16550A
PPP generic driver version 2.4.2
MPPE/MPPC encryption/compression module registered
NET: Registered protocol family 24
PPPoL2TP kernel driver, V0.18.3
PPTP driver version 0.8.5
pflash: found no supported devices
Boot partition size = 262144(0x40000)
Creating 2 MTD partitions on "sflash":
0x00000000-0x00040000 : "pmon"
0x001f0000-0x00200000 : "nvram"
Found a Zentel NAND flash with 2048B pages or 128KB blocks; total size 128MB
lookup_nflash_rootfs_offset: offset = 0x0
nflash: squash filesystem with lzma found at block 9
Creating 2 MTD partitions on "nflash":
0x00000000-0x02000000 : "linux"
0x0013d270-0x02000000 : "rootfs"
NAND device: Manufacturer ID: 0x92, Chip ID: 0xf1 (Zentel NAND 128MiB 3,3V 8-bit)
Bad block table found at page 65408, version 0x01
Bad block table found at page 65344, version 0x01
nand_read_bbt: Bad block at 0x07fe0000
Creating 3 MTD partitions on "brcmnand":
0x00000000-0x02000000 : "trx"
0x02000000-0x04000000 : "jffs2"
0x04000000-0x07f00000 : "brcmnand"
dev_nvram_init: _nvram_init
sdhci: Secure Digital Host Controller Interface driver
sdhci: Copyright(c) Pierre Ossman
u32 classifier
    OLD policer on
Netfilter messages via NETLINK v0.30.
nf_conntrack version 0.5.0 (2048 buckets, 16384 max)
ip_tables: (C) 2000-2006 Netfilter Core Team
net/ipv4/netfilter/tomato_ct.c [Jul 16 2015 23:18:16]
ipt_account 0.1.21 : Piotr Gasidlo <quaker@barbara.eu.org>, http://code.google.com/p/ipt-account/
NET: Registered protocol family 1
NET: Registered protocol family 10
ip6_tables: (C) 2000-2006 Netfilter Core Team
NET: Registered protocol family 17
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
VFS: Mounted root (squashfs filesystem) readonly.
Freeing unused kernel memory: 200k freed
Warning: unable to open an initial console.
Failed to execute /sbin/preinit.  Attempting defaults...
Algorithmics/MIPS FPU Emulator v1.5
ctf: module license 'Proprietary' taints kernel.
et_module_init: passivemode set to 0x0
et_module_init: et_txq_thresh set to 0x400
PCI: Setting latency timer of device 0000:00:01.0 to 64
bcm_robo_enable_switch: EEE is disabled
eth0: Broadcom BCM47XX 10/100/1000 Mbps Ethernet Controller 6.30.163.2002 (r382208)
wl_module_init: passivemode set to 0x0
PCI: Enabling device 0000:01:01.0 (0000 -> 0002)
PCI: Setting latency timer of device 0000:01:01.0 to 64
eth1: Broadcom BCM4331 802.11 Wireless Controller 6.30.163.2002 (r382208)
PCI: Enabling device 0000:02:01.0 (0000 -> 0002)
PCI: Setting latency timer of device 0000:02:01.0 to 64
eth2: Broadcom BCM4360 802.11 Wireless Controller 6.30.163.2002 (r382208)
JFFS2 version 2.2. (NAND) © 2001-2006 Red Hat, Inc.
br0: starting userspace STP failed, staring kernel STP
vlan1: add 33:33:00:00:00:01 mcast address to master interface
vlan1: add 01:00:5e:00:00:01 mcast address to master interface
vlan1: dev_set_promiscuity(master, 1)
device eth0 entered promiscuous mode
device vlan1 entered promiscuous mode
device eth1 entered promiscuous mode
wlc_phy_cal_init_acphy: NOT Implemented
device eth2 entered promiscuous mode
device wl0.1 entered promiscuous mode
br0: port 4(wl0.1) entering listening state
br0: port 3(eth2) entering listening state
br0: port 2(eth1) entering listening state
br0: port 1(vlan1) entering listening state
vlan1: add 33:33:ff:09:15:b8 mcast address to master interface
ADDRCONF(NETDEV_UP): vlan2: link is not ready
br0: port 4(wl0.1) entering learning state
br0: port 3(eth2) entering learning state
br0: port 2(eth1) entering learning state
br0: port 1(vlan1) entering learning state
br0: topology change detected, propagating
br0: port 4(wl0.1) entering forwarding state
br0: topology change detected, propagating
br0: port 3(eth2) entering forwarding state
br0: topology change detected, propagating
br0: port 2(eth1) entering forwarding state
br0: topology change detected, propagating
br0: port 1(vlan1) entering forwarding state
vlan1: add 33:33:00:00:00:02 mcast address to master interface
vlan1: add 33:33:ff:00:00:00 mcast address to master interface