MerlinAU MerlinAU v1.1.4 - The Ultimate Firmware Auto-Updater (Now available in AMTM)

[ExtremeFiretop]

I've just experienced the "high-risk phrases" workflow, and have some gentle suggestions for improvement.

I'd prefer if the change-log scan happened before the postponement period.

Unfortunately this suggestion would require a complete redesign of the workflow compared to our current method.
Will need to think about this more. It probably wouldn't work the way your suggesting but could maybe piggy back on an upcoming feature in v1.1.3

What happened instead is, I had already reviewed the change-log and didn't notice anything of concern or high-risk. After the postponement period had passed, I was puzzled that no update had occurred. I ran the update check to see what was wrong and only then was I told that I need to approve the update.

I would recommend setting up email notifications so you know that a high risk phrases is found in the changelog, and that's the reason the flash didn't happen.
In 1.1.3 the email notification's will contain the changelog contents for review as well.

So I reviewed the change-log again and still could not tell what the high risk item is.

The high risk item is:

- CHANGED: SIP, RTSP and H323 ALG (NAT helpers) are now
disabled by default, as these legacy features tend
to create issues with modern VoIP setups.
This change will only apply to people doing a
factory default reset of their router.

(Also identified with the arrow in the screenshot above)

 

[nlurker]

Please don't redesign everything based on my suggestion. I think I misunderstood how it is intended to work.

The new features in 1.1.3 look like they will help a lot with this situation, along with your suggestion to enable email.

(Perhaps you should call it 1.2, since there is new functionality and it is more than a bug fix.)

 

[ExtremeFiretop]

Please don't redesign everything based on my suggestion. I think I misunderstood how it is intended to work.

The new features in 1.1.3 look like they will help a lot with this situation, along with your suggestion to enable email.

(Perhaps you should call it 1.2, since there is new functionality and it is more than a bug fix.)

This release is slow cookin for sure, but it will have a decent amount of improvements and new features.

As for your suggestion the real issue I see is that currently the postpone stops "all" the logic for an update.
The entire run to flash an update as in, checking memory, setting up directories, downloading the zip, extracting the zip, doing file cleanups, etc all that stuff is held behind the postpone.

If all of a sudden I had to move stuff out from behind the postpone, or move the postpone to a different area of the script it could cause a mess I am not sure I am ready to deal with at the moment lol.
However that being said as you saw above we are working on new log related features. One of which is to download the latest changelog file for review at anytime.

This code works outside the main update logic and is not withheld by the postpone, it also doesn't include all the additional steps for a flash like memory checks and management, file management and downloading large zips for extractions.
Instead this new logic goes online just to grab the single .txt file related to your latest firmware's changelog and nothing else and makes that available to display.

I'm thinking we could maybe make that "arm" of the code be the one to do the changelog checks and it could run independently of the rest of the main update logic without a complete redesign of the flow.
Again I'll need to think more about this, it's not a solid no, but it's also not a solid yes lol. I need to sleep on this suggestion and make some decisions based on complexity and ideal use-cases.

Thanks again for the comment!
I do think you'll benefit from setting up email notifications in AMTM in the long run, and then configuring them in MerlinAU, at least this way if something fails in the future, you'll wake up to a message explaining "why' it did, without a need to troubleshoot.

 

[nlurker]

I agree with everything you said. Moving the changelog check out of the update logic would satisfy my request. It could be triggered immediately when a new release is discovered. The postponed update that comes along later would only need to check if an approval is needed and that I had given it.

 

[E3000]

Are there provisions in place to stop a rogue version of Asuswrt-Merlin (not provided by @RMerlin) slipping through the net?

 

[ExtremeFiretop]

Are there provisions in place to stop a rogue version of Asuswrt-Merlin (not provided by @RMerlin) slipping through the net?

How would a "rogue" update end up on the official repo on sourceforge?

 

[ExtremeFiretop]

Are there provisions in place to stop a rogue version of Asuswrt-Merlin (not provided by @RMerlin) slipping through the net?

And to add to this, even if an unofficial update did end up on the official repo, whether or not you are using this script you would be impacted by such a situation and would likely end up downloading it anyways. (If it did end up happening.)

 

[RMerlin]

Are there provisions in place to stop a rogue version of Asuswrt-Merlin (not provided by @RMerlin) slipping through the net?

I keep a copy of the SHA256 hashes on a totally different location (my website) than the download location to protect against that, as someone would need to hack both my website and my Sourceforge account to be able to slip it through.

 

[dave14305]

And to add to this, even if an unofficial update did end up on the official repo, whether or not you are using this script you would be impacted by such a situation and would likely end up downloading it anyways. (If it did end up happening.)

I keep a copy of the SHA256 hashes on a totally different location (my website) than the download location to protect against that, as someone would need to hack both my website and my Sourceforge account to be able to slip it through.

The “vulnerability” of this script is that it only consults the sha256 hashes contained in the downloaded ZIP file, instead of looking to the asuswrt-merlin.net site as a sanity check. It would be a worthwhile improvement to switch hash source if it is easily parseable from the HTML.

 

[dave14305]

if it is easily parseable from the HTML.

Something like:

curl -s https://www.asuswrt-merlin.net/download | sed -n '/<pre>/,/</pre>/p'

 

[ExtremeFiretop]

MerlinAU Version 1.1.3 released.

What's Changed/Fixed?:

PR: #201 - Node MinorLogicFix and Default Schedule

• This is a fix related to the last release in 1.1.2 where we patched this issue identified below:

-Additionally, the new code for email notifications for nodes could also potentially pause the cron job when sending an email notification.
I did fix it with a quick patch/fix in 1.1.2, but by working around the issue to get a quick fix out the door, the current 1.1.2 release can't process the direct call to process the nodes for example:

I identified the root cause of the issue and resolved it in this PR (#201)

• We also changed the default schedule from: Every SUNDAY:

  0 0 * * Sun

  To EVERYDAY:

  0 0 * * *

The original schedule of every Sunday wasn't ideal for a newbie user that doesn't understand all the features and switches yet.


PR: #202 - General Code improvements.

• Minor code improvements and cleanup in the "_GetAllNodeSettings_ " function. (Thanks @Martinski4GitHub)!

PR: #203 - Code change to address script file path issue

• This PR is all about the issue reported by @aex.perez in this forum post . (Thanks @Martinski4GitHub)!

While we weren't able to replicate the issue, we hope this small change in the logic and order of operations will address your issue. If you notice this issue again please report it.


PR: #204 - Include New Logs Menu

• We added a 15 minute offset to the default schedule if MerlinAU is installed on a node.
  

  if "$inRouterSWmode"
  then
      readonly FW_Update_CRON_DefaultSchedule="0 0 * * *"
  else
      readonly FW_Update_CRON_DefaultSchedule="15 0 * * *"
  fi

• Option to download the latest log file
• Email notification now contains the changelog contents flagged

Designed a new logs menu for log options found below:

PR: #205 - Include Cron Estimates
As requested by @Viktor Jaep and @nlurker in this forum post

We now provide an exact estimate of when the next update will run down to the date and minutes when an update is found.

Example below:

The old method did not factor in the cron job schedule and only factored based on the delay/postpone time. (Found above)


PR: #206 - New menu option to view F/W Update log files.

• Added a new menu option that allows users to review a F/W Update log file. Only the most recent 20 log files are listed for selection. (Thanks @Martinski4GitHub)!

PR: #207 - Minor Wording Cleanup

• Minor change in email notification wording depending if it's installed on a router or node. This was a change recommended by @maghuro in this forum post

if "$inRouterSWmode"
then
printf "\n ${GRNct}em${NOct}.  Toggle F/W Update Email Notifications"
else
printf "\n ${GRNct}em${NOct}.  Toggle F/W Email Notifications"
fi

PR: #208 - Fix Future Estimates

• Redesigned PR: 205 to fix issues with specific crons schedules not being parsed and calculated correctly and be generally cleaner to work on and read.

PR: #209 - Fixes, code cleanup & improvements

• Again, fixed released to PR: 205 to fix issues with specific crons schedules not being parsed and calculated correctly and be generally cleaner to work on and read. (Thanks @Martinski4GitHub)!

PR: #210 - More Code improvements for PR: #205 and PR :#209

• Martinski cleaning up some of his code from PR 209 to make it easier and cleaner to read (Thanks @Martinski4GitHub)!

Just... So much Tightening... So much code... (improvements).
If you haven't noticed already we highly recommend you update ASAP as this includes lots of functional improvements and little bug fixes.


Significant screenshots:

 

[ExtremeFiretop]

Something like:

curl -s https://www.asuswrt-merlin.net/download | sed -n '/<pre>/,/</pre>/p'

@dave14305

I think this is a great idea! As always you've always come in with a helping hand and I much appreciate that.
I already had the other release ready to go, but you can expect this will be investigated for the next release

I think it's a valid point and we can increase the robustness with a "double check" of the sha256 against @RMerlin 's website.
The truth is that however good the idea is, I still think we all agree the risk is fairly low as is. I think that's why you quoted your "vulnerability"

But the point is valid and if we can address it why not, right?
My only counter thought at the moment is, if RMerlin ever has a mismatch between his website and the zip just by accident, it could cause unintended issues.
I'm not sure what his process is in that regard, but I'm sure it's a fairly low risk of that happening.

 

[dave14305]

I still think we all agree the risk is fairly low as is. I think that's why you quoted your "vulnerability"

I feel the probability is low, but the risk is high. I’ve never been fond of SF and all its mirrors and redirects, so I always download from OneDrive and verify hashes from Merlin’s site. But yes, regular users are equally susceptible because most probably never bother to verify hashes at all. So this script can be more secure against a supply chain attack in that regard.

if RMerlin ever has a mismatch between his website and the zip just by accident, it could cause unintended issues.

I would no longer pay attention to the hash inside the zip.

 

[ExtremeFiretop]

I feel the probability is low, but the risk is high. I’ve never been fond of SF and all its mirrors and redirects, so I always download from OneDrive and verify hashes from Merlin’s site. But yes, regular users are equally susceptible because most probably never bother to verify hashes at all. So this script can be more secure against a supply chain attack in that regard.

I would no longer pay attention to the hash inside the zip.

Yes exactly, sorry that is what I meant, obviously the risk involved would be high if any of this did somehow happen and you had some unofficial compromised firmware installed, but the 'probability' is very, very low I'd say as well.
It's not something I've really been concerned of in the time developing and using this script. However no matter how low the probability it is, I wouldn't say impossible.

No matter how secure @RMerlin is the probability is never zero, so I totally understand the viewpoint.
For now I'd say this is probably going to be one of my next priorities for the next release, and shuffle the logic out of the zip and towards the website.

 

[vlord]

Hi @nlurker

Did you have a look at my GitHub page by chance? Or was this a total coincidence that you requested this while I have PR 205 already pending to try and address this? Haha.

I have a PR im working on to factor in the cron in the runtime estimate. Still needs some work but I started it some hours ago sometime this morning

It's on the radar.

I can’t recall if this was asked in the previous 19 pages, but since you are adding this to the text GUI, what are your thoughts on displaying this in the Web GUI somewhere?

 

[nlurker]

I would recommend setting up email notifications so you know that a high risk phrases is found in the changelog, and that's the reason the flash didn't happen.
In 1.1.3 the email notification's will contain the changelog contents for review as well.


The high risk item is:

- CHANGED: SIP, RTSP and H323 ALG (NAT helpers) are now
disabled by default, as these legacy features tend
to create issues with modern VoIP setups.
This change will only apply to people doing a
factory default reset of their router.

If you are including the changelog contents in the email, can you also highlight the high-risk phrases just like you did here?

 

[ExtremeFiretop]

I can’t recall if this was asked in the previous 19 pages, but since you are adding this to the text GUI, what are your thoughts on displaying this in the Web GUI somewhere?

We have no plans currently to work on a webUI for MerlinAU, that would be a separate project down the line if I'm bored and have nothing else to work on maybe lol.
I've played with it once, the HTTP server running on the router is a castrated version of a standard HTTP server, and for good (security) reasons.

 

[ExtremeFiretop]

If you are including the changelog contents in the email, can you also highlight the high-risk phrases just like you did here?

This will be included in the next release and is apart of PR 214: https://github.com/ExtremeFiretop/MerlinAutoUpdate-Router/pull/214
In this code, if it's set as HTML it highlights it yellow:

Else if it's plain text it tags it with asterisks:

 

[ExtremeFiretop]

I feel the probability is low, but the risk is high. I’ve never been fond of SF and all its mirrors and redirects, so I always download from OneDrive and verify hashes from Merlin’s site. But yes, regular users are equally susceptible because most probably never bother to verify hashes at all. So this script can be more secure against a supply chain attack in that regard.

I would no longer pay attention to the hash inside the zip.

This suggestion is now included in PR 214: https://github.com/ExtremeFiretop/MerlinAutoUpdate-Router/pull/214 as well and will be included in the next release.

 

[ExtremeFiretop]

MerlinAU Version 1.1.4 released.

What's Changed/Fixed?:

PR: #213 - Modified top section/header of the Main Menu.

• This PR is for the top section/header of the Main Menu was modified to allow for a bit longer/better description of the listed items. (Thanks @Martinski4GitHub)!

PR: #214 - Update GUI Menu Sizes, Fix Changelog Check, and Email Highlight Changelog Changes, SHA256 changes

• Update GUI Menu Sizes to match Martinskis updates in PR 213,
• Fix Changelog Check which would previously find no high risk phrases for some users.
• MerlinAU will now Highlight the Changelog Changes of high risk in the email as requested by @nlurker here: https://www.snbforums.com/threads/m...dater-now-available-in-amtm.88577/post-908680
• 
• SHA256 changes as requested by @dave14305 here: https://www.snbforums.com/threads/m...dater-now-available-in-amtm.88577/post-908412 to enhance security.

PR: #215 - Minor code fixes & improvements.

• This PR is all about various minor code fixes & improvements. One of which is an improved password debounce when pressing "TAB" key to view the password. (Thanks @Martinski4GitHub)!

Tightening up some code & general code improvements.

Significant screenshots: