Kernel Panic > RT-AC68U -- (multiple areas) . . . not good.

tdhite

Occasional Visitor
Wondering if others are hitting JFFS2 module panics on nvram consolidation. Looked around but only saw a lack of space, but I have plenty (seemingly). Doesn't look like memory uncorrectable error or the like either. Trace below . . . thoughts welcome cuz this one has me mystified at the moment and it's occurring on latest build (and prior as well, but not before).

device: RT-AC68U
firmware version: 386.7_2
nvram show: size: 61148 bytes (4388 left)
jffs:
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mtdblock4 64256 1864 62392 3% /jffs

Panic (reboot log):
May 5 01:05:04 kernel: <6>nvram: consolidating space!
May 5 01:05:04 kernel: <1>Unable to handle kernel NULL pointer dereference at virtual address 00000057
May 5 01:05:04 kernel: <1>pgd = cac54000
May 5 01:05:04 kernel: <1>[00000057] *pgd=9d148031, *pte=00000000, *ppte=00000000
May 5 01:05:04 kernel: <0>Internal error: Oops: 17 [#1] PREEMPT SMP
May 5 01:05:04 kernel: <0>last sysfs file: /sys/class/net/vlan1/brport/port_no
May 5 01:05:04 kernel: <4>CPU: 0 Tainted: P (2.6.36.4brcmarm #1)
May 5 01:05:04 kernel: <4>PC is at jffs2_add_fd_to_list+0x48/0xe0 [jffs2]
May 5 01:05:04 kernel: <4>LR is at jffs2_do_unlink+0x378/0x3c0 [jffs2]
May 5 01:05:04 kernel: <4>pc : [<bf00c480>] lr : [<bf012138>] psr: 20000013
May 5 01:05:04 kernel: <4>sp : cdf95e98 ip : c003b868 fp : 00000000
May 5 01:05:04 kernel: <4>r10: cdf5cc00 r9 : cf441520 r8 : cdfbe260
May 5 01:05:04 kernel: <4>r7 : cfb86a00 r6 : cdfbe275 r5 : cdfbe260 r4 : cdf5cc04
May 5 01:05:04 kernel: <4>r3 : 0059452d r2 : cf441544 r1 : c7b2f936 r0 : 00000047
May 5 01:05:04 kernel: <4>Flags: nzCv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
May 5 01:05:04 kernel: <4>Control: 10c53c7d Table: 9ac5404a DAC: 00000015
May 5 01:05:04 kernel: <0>Process conn_diag (pid: 18167, stack limit = 0xcdf94270)
. . .
May 5 01:05:04 kernel: <4>[<bf00c480>] (PC is at jffs2_add_fd_to_list+0x48/0xe0 [jffs2])
May 5 01:05:04 kernel: <4>[<bf00c480>] (jffs2_add_fd_to_list+0x48/0xe0 [jffs2]) from [<bf012138>] (jffs2_do_unlink+0x378/0x3c0 [jff
May 5 01:05:04 kernel: <4>[<bf012138>] (jffs2_do_unlink+0x378/0x3c0 [jffs2]) from [<bf00a594>] (jffs2_unlink+0x40/0x84 [jffs2])
May 5 01:05:04 kernel: <4>[<bf00a594>] (jffs2_unlink+0x40/0x84 [jffs2]) from [<c00d41f8>] (vfs_unlink+0x80/0xe8)
May 5 01:05:04 kernel: <4>[<c00d41f8>] (vfs_unlink+0x80/0xe8) from [<c00d59f8>] (do_unlinkat+0x148/0x174)
May 5 01:05:04 kernel: <4>[<c00d59f8>] (do_unlinkat+0x148/0x174) from [<c003dd20>] (ret_fast_syscall+0x0/0x30)
May 5 01:05:04 kernel: <0>Code: e5940004 e2844004 e3500000 0a00001c (e5903010)
May 5 01:05:05 kernel: <4>---[ end trace aa8a1d23d4f38ebe ]---
May 5 01:05:05 kernel: <0>Kernel panic - not syncing: Fatal exception
May 5 01:05:05 kernel: <4>[<c0044258>] (unwind_backtrace+0x0/0xf8) from [<c02cf510>] (panic+0x74/0x1a0)
May 5 01:05:05 kernel: <4>[<c02cf510>] (panic+0x74/0x1a0) from [<c0041950>] (die+0x1a4/0x1dc)
May 5 01:05:05 kernel: <4>[<c0041950>] (die+0x1a4/0x1dc) from [<c00453b4>] (__do_kernel_fault+0x64/0x84)
May 5 01:05:05 kernel: <4>[<c00453b4>] (__do_kernel_fault+0x64/0x84) from [<c0045524>] (do_page_fault+0x150/0x1ec)
May 5 01:05:05 kernel: <4>[<c0045524>] (do_page_fault+0x150/0x1ec) from [<c003d3a4>] (do_DataAbort+0x30/0x9c)
May 5 01:05:05 kernel: <4>[<c003d3a4>] (do_DataAbort+0x30/0x9c) from [<c03844cc>] (__dabt_svc+0x4c/0x60)
May 5 01:05:05 kernel: <4>Exception stack(0xcdf95e50 to 0xcdf95e98)
May 5 01:05:05 kernel: <4>5e40: 00000047 c7b2f936 cf441544 0059452d
May 5 01:05:05 kernel: <4>5e60: cdf5cc04 cdfbe260 cdfbe275 cfb86a00 cdfbe260 cf441520 cdf5cc00 00000000
May 5 01:05:05 kernel: <4>5e80: c003b868 cdf95e98 bf012138 bf00c480 20000013 ffffffff
May 5 01:05:05 kernel: <4>[<c03844cc>] (__dabt_svc+0x4c/0x60) from [<bf00c480>] (jffs2_add_fd_to_list+0x48/0xe0 [jffs2])
May 5 01:05:05 kernel: <4>[<bf00c480>] (jffs2_add_fd_to_list+0x48/0xe0 [jffs2]) from [<bf012138>] (jffs2_do_unlink+0x378/0x3c0 [jff
May 5 01:05:05 kernel: <4>[<bf012138>] (jffs2_do_unlink+0x378/0x3c0 [jffs2]) from [<bf00a594>] (jffs2_unlink+0x40/0x84 [jffs2])
May 5 01:05:05 kernel: <4>[<bf00a594>] (jffs2_unlink+0x40/0x84 [jffs2]) from [<c00d41f8>] (vfs_unlink+0x80/0xe8)
May 5 01:05:05 kernel: <4>[<c00d41f8>] (vfs_unlink+0x80/0xe8) from [<c00d59f8>] (do_unlinkat+0x148/0x174)
May 5 01:05:05 kernel: <4>[<c00d59f8>] (do_unlinkat+0x148/0x174) from [<c003dd20>] (ret_fast_syscall+0x0/0x30)
May 5 01:05:05 kernel: _ End of Reboot message (8180) ... _______________________________________________________
 

RMerlin

Asuswrt-Merlin dev
Could be filesystem corruption. Make a backup of the JFFS partition, reformat it, then restore the content.
 

tdhite

Occasional Visitor
Could be filesystem corruption. Make a backup of the JFFS partition, reformat it, then restore the content.
Thanks. One of two things was the culprit (the pity is I made two changes as opposed to one at a time . . . bad practice, but we all 'whiff' once in a while).

1) Reformatted JFFS.
2) Put in a post-mount script handler to move the conn_diag database onto USB.

Issue is solved. Over time I'll drop the post-mount script and see if there's a conn_diag and nvram consolidate race condition of some sort. Can't be sure of either that or /jffs file system issue.

Hopefully helps the next person if another sees the panics.
 

tdhite

Occasional Visitor
Nope, declared victory too early . . . just panicked again. Back to the drawing board. This panic is different -- within wlc_bss_mac_rxframe_event.

Here's a stack trace for those interested -- at this point I believe there is a potential spurious hardware failure bogus-ly generating an interrupt or similar. Not sure, but it isn't JFFS after all. It's looking random, which is very hard to track in a kernel.


May 5 01:05:04 kernel: _ Start of Reboot message (8180) ... _______________________________________________________
May 5 01:05:04 kernel: jbd bf5d8000 423 (2.6.36.4brcmarm #1)
May 5 01:05:04 kernel: <4>P?U? at __kmalloc+0x70/0x104
May 5 01:05:04 kernel: <4>LR is at __kmalloc+0x1c/0x104
May 5 01:05:04 kernel: <4>pc bf5b7000 19855
May 5 01:05:04 kernel: <4>module: sd_m4>] psr: 20000193
May 5 01:05:04 kernel: <4>sp : cacscsi_wait_scan bf5a5000 416
May 5 01:05:04 kernel: <4000
May 5 01:05:04 kernel: <4>r10: 0000002c r9 : 000008738

. . .

May 5 01:05:04 kernel: <4>[<bf1a3e8c>] (wlc_bss_mac_1fb0
May 5 01:05:04 kernel: <0>1e60: c03cd444 bed3d4d8 c8225868 cac51e80 c007782c c03754b8 20000093 ffffffff
May 5 01:05:04 kernel: <0>1e80: cd103380 00000000 cac50000 cf823600 00000011 cac50000 c040b588 c0cv+0x2c3c/0x2e48 [wl])
May 5 01:05:04 kernel: <4>unwind: Index not found bf254fc8
May 5 01:05:04 kernel: <0>Co51fb0 00000011 c03cd420
May 5 01:05:04 kernel: <0>1ec0: cada6000 cf823600 00000011 cac50000 c040b588 00000000 cac51fb0 c03cd444
May 5 01:05:04 kernel: <0>1ee0: bed3d4d8 c0060d40 cac51fb0 cd0a1820 cf8237a0 cf823798 cf51b088 00000000
May 5 01:05:04 kernel: <0>1f00: 00000000 cf8236f0 00000020 00000000 cdedde00 00000011 00000000x74/0x1a0) from [<c0041958>] (die+0x1ac/0x1dc)
May 5 01:05:04 kernel: <4>[<c0041958>] (die+0x1ac/0x1dc) from [<c00453b4>] (__do_kernel_fault+0x64/0x8400000005 cecc31c0 c00c8928 00000ault+0x64/0x84) from [<c0045524>d0a1460 cac51f90 bed3d518 cac51f90 0000002a c003dec8 00063860
May 5 01:05:04 kernel: <0>1f80: 00093410 000922a4 00000002 c003dec8 00000000 c0041208 00000000 00000000
May 5 01:05:04 kernel: <0>1fa0: 000922a4 c003dd20 00063860 00093410 00093410 000922fc 00000000 0009230c
May 5 01:05:04 kernel: <0>1fc0: 00063860 00093410 000922a4 00000002 000922fc 00000000 bed3d510 0000054c
May 5 01:05:04 kernel: <0>1fe0: 00090fc4 bed3d4d8 0002d33c 4014aac8 60000010 00093410 00001000 00000000
May 5 01:05:04 kernel: <4>[<c00c46d8>] (PC is at __km00c4684 c00c46d8 20000193 ffffff] (__kmalloc+0x70/0x104) from [<c01d9cc8>] (osl_malloc+0x18/0x58)
May 5 01:05:04 kernel: <4>[<c01d9cc8>] (osl_malloc+0x18/0x58) from [<bf06be28>] (bcm_mp_alloc+0x24/0x88 [wl])
May 5 01:05:04 kernel: <4>[<bf06be28>] (bcm_mp_alloc+0x24/0x88x18/0x58) from [<bf06be28>] (bcm_mp_alloc+0x24/0x88 [wl])
May 5 01:05:04 kernel: <4>[<b204b88>] (wlc_event_alloc+0xc/0x24 [wl]) from [<bf1a3e8c>] (wlc_bss_mac_rxframe_event+0x40/0x170 [wl])
May 5 01:05:04 kernel: <4>[<bf1a3e8c>] (wlc_bss_mac_rxframe_event+0x40/0x170 [wla3ff8>] (wlc_bss_mac_event+0x3c/0x44 [wl]) from [<bf254fc8>] (wlc_recv+0x2c3c/0x2e48 [wl])
May 5 01:05:04 kernel: <4>unwind: Index not found bf254fc8
May 5 01:05:04 kernel: <0>Code: e083c002 e3580000 0a00001c e5941010 (e7981001)
May 5 01:05:05 kernel: <4>---[ end trace 328c85e20eff5c2e ]---
May 5 01:05:05 kernel: _ End of Reboot message (8180) ... _______________________________________________________
 

Similar threads

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top