Mounting issues with add-ons

[DanNuke86]

Hi,

Wondering if anyone can help with my query. I've read through similar issues others have had but can't resolve my issues.

There is an intermittent issue with the add-ons mounting. I run Division, Unbound, Conmon, Modmon, ScMerlin, SpdMerlin, NtpMerlin and FlexQoS.

Sometimes when I reboot, none of the add-ons will mount / add-ons tab does not show. Doesn't seem to be a time on it resolving itself.

The only way I find it 'sorts itself' is to keep rebooting until it works. Sometimes there may be one of them missing (usually unbound) and I just reinstall it in AMTM.

Can anyone point me in the right direction to resolving this so the add-ons mount every time? I've had this issue right through the alpha and beta firmware and carried out multiple hard resets to try and resolve this.

 

[L&LD]

Update/Reset Mini Guide

amtm Step-by-Step https://www.snbforums.com/threads/amtm-step-by-step-install-guide-l-ld.56237/#post-483421

(Note; do not try to install amtm, it is already baked into the firmware since RMerlin v384.15_0).

Do not use a previously used amtm drive in a freshly reset router before formatting the USB drive in a PC with NTFS file format.

 

[JGrana]

Curious. How quickly after a reboot do you log back into the WebUI?
Try this - reboot - but don't log in for 5 minutes or so.

 

[dave14305]

There is an inherent risk in the way addons add themselves to the webui, since they can tend to start simultaneously and all will try to make binding mounts and editing of the same file at the same time. It’s probably a risk of running several addons. It’s a bit of a free-for-all but we all follow the same steps documented in the API.

 

[DanNuke86]

Curious. How quickly after a reboot do you log back into the WebUI?
Try this - reboot - but don't log in for 5 minutes or so.

I’ve not tried it so recently, but I’ve reset it once before going to bed and it hadn’t shown up by the morning so had to reset again. Feel like sometimes it depends on how quick the NTP syncs..

 

[DanNuke86]

Curious. How quickly after a reboot do you log back into the WebUI?
Try this - reboot - but don't log in for 5 minutes or so.

I’ve not tried re-formatting it from the pc, but have always formatted it via amtm, after doing a hard reset before I start installing the add-ons. Does this make the difference with the issue I’m experiencing?

 

[Jack Yaz]

There is an inherent risk in the way addons add themselves to the webui, since they can tend to start simultaneously and all will try to make binding mounts and editing of the same file at the same time. It’s probably a risk of running several addons. It’s a bit of a free-for-all but we all follow the same steps documented in the API.

I stagger each of mine with a short sleep to try and avoid multiple processes sed'ing the MenuTree.js file. Currently I've not had any problems with WebUI reappearing running all of my scripts, as well as FlexQoS . I do not use Unbound's Stats, Skynet or Diversion's WebUi tho, I'd be interested to know it those who have issues are using either of those?

 

[Jack Yaz]

Curious. How quickly after a reboot do you log back into the WebUI?
Try this - reboot - but don't log in for 5 minutes or so.

I wonder if its worth me posting my "ntp boot watchdog" script. I noticed NTP wouldn't reliably start with my 86U sometimes so I made a script thst aggressively has it start

 

[JGrana]

I stagger each of mine with a short sleep to try and avoid multiple processes sed'ing the MenuTree.js file. Currently I've not had any problems with WebUI reappearing running all of my scripts, as well as FlexQoS . I do not use Unbound's Stats, Skynet or Diversion's WebUi tho, I'd be interested to know it those who have issues are using either of those?

I am using Unbound stats - and typically the Unbound tab always shows up.
The last few times I experienced the missing Addons, Unbound and ScMerlin tabs are present, connmon, spdMerlin and ntpMerlin not.
I did notice in post-mount that the sequence is Unbound, uiScribe, scMerlin, spdMerlin, connmon, ntpMerlin and another scMerlin.
I also think uiScribe runs fine (at least when I checked).

The missing tabs doesn’t happen all the time - in my case usually after a firmware update.
A reboot seems to bring them back once I experience the issue.

 

[JGrana]

I’ve not tried re-formatting it from the pc, but have always formatted it via amtm, after doing a hard reset before I start installing the add-ons. Does this make the difference with the issue I’m experiencing?

I wouldn’t think so. Since you can successfully install Entware then all the Addons after the amtm format/reboot, the disks file system sounds fine.
The missing tabs feels like a startup race condition. And in my case is very intermittent, typically after a firmware update. The router is likely very busy for a while after an update.

 

[Jack Yaz]

I am using Unbound stats - and typically the Unbound tab always shows up.
The last few times I experienced the missing Addons, Unbound and ScMerlin tabs are present, connmon, spdMerlin and ntpMerlin not.
I did notice in post-mount that the sequence is Unbound, uiScribe, scMerlin, spdMerlin, connmon, ntpMerlin and another scMerlin.
I also think uiScribe runs fine (at least when I checked).

The missing tabs doesn’t happen all the time - in my case usually after a firmware update.
A reboot seems to bring them back once I experience the issue.

Next time it happens, can you check if there are multiple entries for menuTreee in df -h please?

 

[JGrana]

Next time it happens, can you check if there are multiple entries for menuTreee in df -h please?

Sure. It happens very infrequently but I will check as soon as it does!

 

[DanNuke86]

I wonder if its worth me posting my "ntp boot watchdog" script. I noticed NTP wouldn't reliably start with my 86U sometimes so I made a script thst aggressively has it start

It tends to be be ntpMerlin or unbound that don’t always load for me. There’s no consistent trend with the intermittent issue. It does seem like something is interfering with the mount. When the ntp sync takes a while, this usually is when the mounts don’t happen or the add-one tab doesn’t appear. Not sure what (if anything) is interfering first though..

How does your script work that you mentioned?

 

[DanNuke86]

Next time it happens, can you check if there are multiple entries for menuTreee in df -h please?

Is this something I could check too? Is that command done in SSH?

 

[Jack Yaz]

Is this something I could check too? Is that command done in SSH?

yes and yes!

 

[DanNuke86]

yes and yes!

Did a reboot just now and left it 15 mins before logging in to the web gui to find only Modmon has loaded this time.

Was it just the df -h command in SSH? Did that and the below screenshot is what it showed up.

 

[Jack Yaz]

Did a reboot just now and left it 15 mins before logging in to the web gui to find only Modmon has loaded this time.

Was it just the df -h command in SSH? Did that and the below screenshot is what it showed up.

Hm. If possible can you share a copy of syslog covering the reboot?

 

[DanNuke86]

Hm. If possible can you share a copy of syslog covering the reboot?

There's quite a lot of text lines, is there a particular section to paste, or just all of it from the reboot command?

 

[Jack Yaz]

There's quite a lot of text lines, is there a particular section to paste, or just all of it from the reboot command?

all of it please. if you prefer, please upload as unlisted to pastebin and PM me a link

 

[Jack Yaz]

Here's the problem

run_custom_script("post-mount", 120, mountpoint, NULL);

The firmware gives us 2 minutes to complete post-mount or it gets timed out, I think. That means S61unbound and S77ntpd/S77chronyd (and any other S* scripts that wait for NTP) which wait for up to 5 minutes will actually be killed if NTP hasn't started within 2 minutes of post-mount being called. I haven't seen this due to my aggressive NTP syncing on boot (which I will share shortly if my current fix idea for this is ineffective)