Why does watchdog keep restarting httpd?

[Jerry12]

Ever since installing ASUS firmware 3.0.0.4.378_9529 on RT-AC3200, it fills the log with messages like:

Apr 20 13:22:11 watchdog: restart httpd
Apr 20 13:22:11 rc_service: watchdog 483:notify_rc start_httpd
Apr 20 13:22:11 RT-AC3200: start httpd - SSL
Apr 20 13:22:41 watchdog: restart httpd
Apr 20 13:22:41 rc_service: watchdog 483:notify_rc start_httpd
Apr 20 13:22:41 RT-AC3200: start httpd - SSL
Apr 20 13:23:11 watchdog: restart httpd
Apr 20 13:23:11 rc_service: watchdog 483:notify_rc start_httpd
Apr 20 13:23:11 RT-AC3200: start httpd - SSL
Apr 20 13:23:41 watchdog: restart httpd
Apr 20 13:23:41 rc_service: watchdog 483:notify_rc start_httpd
Apr 20 13:23:41 RT-AC3200: start httpd - SSL​

A few forum threads mention such messages but their tips didn't fix it: Reinstall the firmware. Look for a long client name that might somehow crash the HTTP server. I did the firmware-installation dance twice now: unmount & disconnect any USB drives, reboot, flash firmware, power cycle, configure, reconnect USB drives.

It's not obvious whether httpd is crashing or watchdog is killing and restarting it.

The router is set up for HTTPS admin. On HTTP the router merely returns a "Settings have been updated" page. Does it use the same httpd to serve both HTTP and HTTPS?

The router is configured simply: WiFi, AiProtection, SSH, Firewall, DHCP.
No VPN, SmartConnect, IPv6, etc. I've even removed the USB drive and turned off those services.

Debugging ideas?

 

[RMerlin]

Test with fewer connected clients, and add them one by one until you track down any client causing it.

You can also try the latest beta firmware from Asus, it might have resolved the issue already.

http and https is handled by the same program, which is run twice - once per interface.

 

[Jerry12]

Great idea. I'll try that.

Does it matter if they're connected by ethernet or WiFi?

 

[RMerlin]

Great idea. I'll try that.

Does it matter if they're connected by ethernet or WiFi?

Doubtful. But since we don't know yet what is causing your crashes, can't rule out either of these.

 

[sfx2000]

There was another thread where update forced a factory reset of the device, as the httpd needed to get at files that it couldn't, thus triggering the reboot...

/*
task takes to long - kick the dog
yipe yipe yipe...
*/

 

[Jerry12]

Test with fewer connected clients, and add them one by one until you track down any client causing it.

I disconnected clients one by one, checking the logs, until it was down to my laptop (on just one of Ethernet or WiFi) and the thermostat. The problem continued.

You can also try the latest beta firmware from Asus, it might have resolved the issue already.

After upgrading to Beta Version 9.0.0.4.380.2695, the log is no longer showing this symptom! But it now takes several minutes to get a page from the web interface. That was not happening before.

 

[Jerry12]

update forced a factory reset of the device, as the httpd needed to get at files that it couldn't, thus triggering the reboot...

Did the update auto-trigger a factory reset or did the update force the sysadmin to do a factory reset for it to run OK?

 

[sfx2000]

Did the update auto-trigger a factory reset or did the update force the sysadmin to do a factory reset for it to run OK?

RMerlin would know all the details - but if I recall correctly - there was a release that forced a rebuild of the file system, and this was a hard action at the local device - e.g. mash the button...

 

[Jerry12]

In this case does it work to do a factory reset then restore a saved config?

Thanks, @sfx2000 ! Why doesn't ASUS reveal which firmware upgrades require factory reset?

 

[jitender singh]

just in case if someone finds it useful -
I came across the same issue where httpd was being restarted by watchdog, this happened every time I clicked on "Clients" or "View (client) List" button on the index.asp page.
Installed strace via asus tools (ipkg install strace), and figured that the httpd process was looking for a directory /jffs/usericon/ . (My guess is, I might have assigned a custom icon to one of the client devices).
/jffs was showing up as empty readonly directory.

Solution:
Go to Administration -> System. Select yes for 'Format JFFS partition at next boot'
Apply and reboot.
Once booted, login via ssh and create the usericon directory. (mkdir /jffs/usericon/ )
Reboot again.

 

[RMerlin]

just in case if someone finds it useful -
I came across the same issue where httpd was being restarted by watchdog, this happened every time I clicked on "Clients" or "View (client) List" button on the index.asp page.
Installed strace via asus tools (ipkg install strace), and figured that the httpd process was looking for a directory /jffs/usericon/ . (My guess is, I might have assigned a custom icon to one of the client devices).
/jffs was showing up as empty readonly directory.

Solution:
Go to Administration -> System. Select yes for 'Format JFFS partition at next boot'
Apply and reboot.
Once booted, login via ssh and create the usericon directory. (mkdir /jffs/usericon/ )
Reboot again.

Good catch - fixed.

Note that this doesn't mean that every httpd crashes were caused by it - it's a fairly unusual situation when the jffs partition isn't properly mounted.