Scribe AX86U - 386.1 - Logging seems to be crashing

[JR Godwin]

Hi,

Running an AX86U on latest Merlin release firmware 386.1, and router had been running for approximately 6 days when logging into syslog seems to have crashed out. I didn't notice it until today, so I rebooted the router, and looking through the latest logs, seems to be crashing out again. I also noticed that the headers for the logging lines kind of changed between when it was last running on the 12th, versus how it was running today on the 15th....

EDIT - Just noticed there's a newer release out. Going to update router to that and see what happens.

From then....

Spoiler

Feb 12 01:59:07 RT-AX86U-B330 kernel: potentially unexpected fatal signal 11.
Feb 12 01:59:07 RT-AX86U-B330 kernel: CPU: 1 PID: 6837 Comm: dcd Tainted: P O 4.1.52 #2
Feb 12 01:59:07 RT-AX86U-B330 kernel: Hardware name: Broadcom-v8A (DT)
Feb 12 01:59:07 RT-AX86U-B330 kernel: task: ffffffc032506b40 ti: ffffffc024050000 task.ti: ffffffc024050000
Feb 12 01:59:07 RT-AX86U-B330 kernel: PC is at 0xf703139c
Feb 12 01:59:07 RT-AX86U-B330 kernel: LR is at 0x1dd58
Feb 12 01:59:07 RT-AX86U-B330 kernel: pc : [<00000000f703139c>] lr : [<000000000001dd58>] pstate: 600f0010
Feb 12 01:59:07 RT-AX86U-B330 kernel: sp : 00000000ff7fd858
Feb 12 01:59:07 RT-AX86U-B330 kernel: x12: 00000000000a2050
Feb 12 01:59:07 RT-AX86U-B330 kernel: x11: 00000000f60ff024 x10: 00000000000a23c4
Feb 12 01:59:07 RT-AX86U-B330 kernel: x9 : 00000000f60ff75c x8 : 00000000000a287c
Feb 12 01:59:07 RT-AX86U-B330 kernel: x7 : 00000000f60ff794 x6 : 00000000000a2876
Feb 12 01:59:07 RT-AX86U-B330 kernel: x5 : 0000000000000000 x4 : 00000000f60ff740
Feb 12 01:59:07 RT-AX86U-B330 kernel: x3 : 0000000000000000 x2 : 00000000ff7fd834
Feb 12 01:59:07 RT-AX86U-B330 kernel: x1 : 000000000007d652 x0 : 0000000000000000
Feb 12 02:00:08 RT-AX86U-B330 kernel: echo (11452): drop_caches: 3
Feb 12 02:00:09 RT-AX86U-B330 kernel: CONSOLE: 711457.232 wl1: r0hole: tx_exp_seq 0x9f, seq 0xa0
Feb 12 02:00:13 RT-AX86U-B330 kernel: sd 0:0:0:0: [sda] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Feb 12 02:00:13 RT-AX86U-B330 kernel: sd 0:0:0:0: [sda] Sense Key : Medium Error [current]
Feb 12 02:00:13 RT-AX86U-B330 kernel: sd 0:0:0:0: [sda] Add. Sense: Unrecovered read error
Feb 12 02:00:13 RT-AX86U-B330 kernel: sd 0:0:0:0: [sda] CDB: Read(10) 28 00 01 8a d0 18 00 00 08 00
Feb 12 02:00:13 RT-AX86U-B330 kernel: blk_update_request: critical medium error, dev sda, sector 25874456
Feb 12 02:00:15 RT-AX86U-B330 rc_service: service 11666:notify_rc restart_dnsmasq
Feb 12 02:00:15 RT-AX86U-B330 custom_script: Running /jffs/scripts/service-event (args: restart dnsmasq)
Feb 12 02:00:18 RT-AX86U-B330 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Feb 12 02:00:18 RT-AX86U-B330 custom_script: Running /jffs/scripts/dnsmasq.postconf (args: /etc/dnsmasq.conf)
Feb 12 02:00:20 RT-AX86U-B330 Diversion: restarted Dnsmasq to apply settings
Feb 12 02:00:20 RT-AX86U-B330 uiDivStats: dnsmasq has restarted, restarting taildns
Feb 12 02:00:25 RT-AX86U-B330 kernel: sd 0:0:0:0: [sda] FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Feb 12 02:00:25 RT-AX86U-B330 kernel: sd 0:0:0:0: [sda] Sense Key : Medium Error [current]
Feb 12 02:00:25 RT-AX86U-B330 kernel: sd 0:0:0:0: [sda] Add. Sense: Unrecovered read error
Feb 12 02:00:25 RT-AX86U-B330 kernel: sd 0:0:0:0: [sda] CDB: Read(10) 28 00 01 86 ad 30 00 00 08 00
Feb 12 02:00:25 RT-AX86U-B330 kernel: blk_update_request: critical medium error, dev sda, sector 25603376

To now...

Spoiler

Feb 15 10:50:15 kernel: EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: user_xattr
Feb 15 11:50:15 custom_script: Running /jffs/scripts/post-mount (args: /tmp/mnt/USBAX86U)
Feb 15 10:50:15 kernel: CONSOLE: 146919.925 wl1.0: wlc_send_bar: for 98:5f:d3:db:60:30 seq 0x1 tid 3
Feb 15 10:50:15 kernel: Adding 2097148k swap on /tmp/mnt/USBAX86U/myswap.swp. Priority:-1 extents:6 across:2260988k
Feb 15 11:50:15 Diversion: Starting Entware and Diversion services on /tmp/mnt/USBAX86U
Feb 15 10:50:16 dhcp_client: bound 158.140.229.166/255.255.255.0 via 158.140.229.1 for 1800 seconds.
Feb 15 10:50:16 custom_script: Running /jffs/scripts/service-event (args: start samba)
Feb 15 10:50:16 dnsmasq[1242]: exiting on receipt of SIGTERM
Feb 15 10:50:16 custom_config: Appending content of /jffs/configs/dnsmasq.conf.add.
Feb 15 10:50:16 custom_script: Running /jffs/scripts/dnsmasq.postconf (args: /etc/dnsmasq.conf)
Feb 15 10:50:16 kernel: klogd: exiting
Feb 15 10:50:16 syslogd exiting

Seems the Router name has dropped out...and, I really don't know what that IP is for that dhcp_client message.
That's where the logging has ended and nothing more coming up.

I'm running the majority of the amtm stuff minus yazfi and qos stuff, and also vpnmgr and YazDHCP.

 

[dave14305]

I'm running the majority of the amtm stuff minus yazfi and qos stuff, and also vpnmgr and YazDHCP.

Since you’re asking about a syslog problem, are you also running Scribe which is a syslog replacement? It’s a pretty important detail to include to get accurate help.

 

[JR Godwin]

Hi, Yes. As I said, pretty much the whole amtm set with the exceptions I noted.

 

[JR Godwin]

So, should I push this over to the addons section?

 

[elorimer]

You have three separate things going on. The first is that I'm pretty sure you are running syslog-ng, because of the final two messages about syslogd and klogd exiting in your second log. That's normal and not a sign of syslog crashing out; it is existing and syslog-ng is taking over. If that is the end of your logging, then you have a Scribe/syslog-ng problem and you should move it to addons and tag it for Scribe.

The second is whether or not the host is appearing in the log messages in the second log. That may just be a template question if it is scribe, but it looks like those messages are from before syslog-ng is running; the question would be what the lines look like after.

The third is that first log looks like it has ugly stuff in it that makes it look like your USB stick is borked.

 

[JR Godwin]

Is that a technical term? Swedish Chef? Like physical errors or a problem with it's formatting? I'll move it over to the addons side....or is there a mod who could move the thread over as is?

 

[L&LD]

@JR Godwin, you need to ask either @thiggins or @RMerlin in a PM for that.