Seeking Feedback & Contributions: Merlin Auto Update Solutions

[bbunge]

Who is this script for?

This script is for the person who wants to use Merlin firmware but is too lazy to manage it. Just the sort who will get in trouble. Actually, they will get into trouble anyway, brick their router and spend money on another router.

 

[ExtremeFiretop]

If there's no appetite for a router based script, I can just drop it.

I'll continue to develop the desktop based one, because I do have a use case for it to install my certs into NGINX when they renew... (Flashing firmware is a skippable option in that one).

But the router based one can be dropped if people don't have any use case. It was an idea to table, and I rather drop it earlier than later down the path honestly.

 

[bbunge]

If there's no appetite for a router based script, I can just drop it.

I'll continue to develop the desktop based one, because I do have a use case for it to install my certs into NGINX when they renew... (Flashing firmware is a skippable option in that one).

But the router based one can be dropped if people don't have any use case. It was an idea to table, and I rather drop it earlier than later down the path honestly.

Do you really trust a Windows PC to update your Linux router?

Play with it for yourself and keep it for yourself. But please, do not share it! Most of the "Old Hats" here will thank you...

 

[ExtremeFiretop]

Do you really trust a Windows PC to update your Linux router?

Play with it for yourself and keep it for yourself. But please, do not share it! Most of the "Old Hats" here will thank you...

Kinda an odd question lol.

You realize the normal update process is done through a browser right? Can be done on Mac OS? Windows? Linux? Regardless of OS?

Do I trust my Windows PC to update my Router... well yes. Since the only other alternative is for the router to update itself.

 

[RMerlin]

Blindly writing a firmware image to flash is a bad idea for many reasons. I haven`t looked at how this is implemented on your end, but here are a few important factors to keep in mind.

1) If the router is a single image model, i.e. any model that uses a .trx file, then writing directly to the partition will have very high risk of the router crashing during the reboot, because you are writing to the filesystem that is currently running the OS. Asus has a whole procedure when updating the firmware that iincludes creating a temp rootfs, copying some critical components to it, and switching to it before writing the firmware image.

2) On multi-image models, are you making sure you are writing to the second partition, and switching to it? The whole procedure is closed source, so I have no idea where and how Asus handles switching partition. If you are writing to the current live partition, same issue as 1) may occur with crashes.

3) What validation are you doing to ensure that the firmware image hasn`t been tampered in transit? Asus uses an RSA signature on the image in their live update procedure. There is no such thing with Asuswrt-Merlin firmware images

4) What validation are you doing to ensure the firmware image was properly uploaded to the router? The firmware update process has a checksum check that ensures the file was properly transferred

5) What validation are you doing to ensure the correct model is being uploaded? The firmware update process has a header check that ensures you aren`t trying to flash the wrong image. I don't know if hnd-write handles that, or if it's a separate pre-upgrade check, as the upgrade code has been closed source for many years now.

6) What validation are you doing to ensure there are no pre-requisite before flashing a firmware? While uncommon, there has been a number of scenarios over the years where a minimal firmware version was required before upgrading to the next. Asus' liveupdate has such checks in place, there's no such thing with Asuswrt-Merlin. For example, when going from an older 32 MB firmware partition to 64 MB around the xxx.55 era for the RT-AC68U


This is why I never provide support when a user asks on how to manually write a firmware image to their router, because there are many gotchas that can make things go wrong.

 

[ExtremeFiretop]

Blindly writing a firmware image to flash is a bad idea for many reasons. I haven`t looked at how this is implemented on your end, but here are a few important factors to keep in mind.

1) If the router is a single image model, i.e. any model that uses a .trx file, then writing directly to the partition will have very high risk of the router crashing during the reboot, because you are writing to the filesystem that is currently running the OS. Asus has a whole procedure when updating the firmware that iincludes creating a temp rootfs, copying some critical components to it, and switching to it before writing the firmware image.

2) On multi-image models, are you making sure you are writing to the second partition, and switching to it? The whole procedure is closed source, so I have no idea where and how Asus handles switching partition. If you are writing to the current live partition, same issue as 1) may occur with crashes.

3) What validation are you doing to ensure that the firmware image hasn`t been tampered in transit? Asus uses an RSA signature on the image in their live update procedure. There is no such thing with Asuswrt-Merlin firmware images

4) What validation are you doing to ensure the firmware image was properly uploaded to the router? The firmware update process has a checksum check that ensures the file was properly transferred

5) What validation are you doing to ensure the correct model is being uploaded? The firmware update process has a header check that ensures you aren`t trying to flash the wrong image. I don't know if hnd-write handles that, or if it's a separate pre-upgrade check, as the upgrade code has been closed source for many years now.

6) What validation are you doing to ensure there are no pre-requisite before flashing a firmware? While uncommon, there has been a number of scenarios over the years where a minimal firmware version was required before upgrading to the next. Asus' liveupdate has such checks in place, there's no such thing with Asuswrt-Merlin. For example, when going from an older 32 MB firmware partition to 64 MB around the xxx.55 era for the RT-AC68U


This is why I never provide support when a user asks on how to manually write a firmware image to their router, because there are many gotchas that can make things go wrong.

1. Writing to the Second Partition and Switching:

The script doesn't provide any explicit mechanism to ensure that the firmware is being written to the second partition, nor does it handle switching between partitions. It utilizes the hnd-write command to flash the firmware, but there's no partition handling within the provided script. It would require a more intimate understanding of the router's firmware update procedure. If the router's system provides a command or API to switch partitions and write to a specific partition, these capabilities should be utilized within the script. I realize you say it's closed source so that may be easier said than done. With modern tools like Ghidra we could try Decompilation instead of disassembly to understand more? Creating a mechanism to switch to the inactive partition, write the new firmware there, and then switch back might be a way to address this concern.

2. Validation against Tampering:

The script includes a checksum validation by comparing the hash of the downloaded firmware file against the expected checksum provided in the sha256sum.sha256 file. This helps to ensure the firmware image hasn't been tampered with in transit on the download to the device. However, it doesn't re-test at the router, I can implement a secondary checksum validation on the upload as well?

3. Validation of Proper Upload:

The script doesn't provide a distinct validation to ensure that the firmware image was properly uploaded to the router aside from the command closing without error (try catch block). There's no additional checksum verification performed post-upload. That can be added easiler. I can also add a check to confirm the firmware upload didn't just closed without error but also closed with a result of 100%.

4. Validation of Correct Model:

The desktop script relies on user input of the router model, and based on that input, fetches matching firmware and uploads that to the router. The router based script does a MODEL=$(nvram get model) and similary parses SourceForge for the matching firmware model. If it's not a perfect match it doesn't download anything and fails outright before even trying to flash anything. Implementing a header check in the script to verify the model information in the firmware image against the model information of the router before proceeding with the upload could address this. This would require the firmware images to have identifiable headers with model information, and a way to retrieve the model information from the router.

5. Validation of Pre-requisites before Flashing:

The script doesn’t contain any mechanism to check for pre-requisite firmware versions before flashing a new firmware. The absence of such a validation could potentially lead to issues if there are required intermediate firmware versions. This was something I wasn't even aware of at all... To be honest this is as much of a learning experence than anything, but what I can say is i actually have been using this to flash my firmware since March of this year (Just without a GUI which I recently added) and never had a crash, always a successful upload and flash and reboot to the latest version of your firmware. Now I realize more can be improve to help secure those steps.

But my thought was pretty simple, if a browser can initial a flash when it receives an update signal, surely we can engage the same commands via a different method?

 

[RMerlin]

But my thought was pretty simple, if a browser can initial a flash when it receives an update signal, surely we can engage the same commands via a different method?

The router does a lot more work than just writing with hnd-write. If you look at do_upgrade_post() within httpd/web.c, you will get a glimpse of things it does when a firmware image gets uploaded to the router.

I have no idea what's involved in Asus' LiveUpdate since I haven't looked at that code in years, but it's probably equally complex.

 

[Tech9]

I believe @RMerlin posts just ended the project. Otherwise one of the requirements must be a new router available.

 

[Ranger802004]

I did find references to: mtd-write2 but none of my 2 routers (GT-AXE11000 and RT-AX88U) or my buddies router recognize it.

The hnd-write seems to be the right command so far, the PowerShell accounts for the different models by the filename variable, the filename variable would have a model name, whichever would be selected further up the script by the user. In the router based script it uses the model name itself to identify the firmware to download and flash it via the hnd-write command.

But if hnd-write is the wrong command for any models I can put a if statement for those models and a different flash command.

That's because the older routers use it, this probably if written would have to be a shell script that executes on the router to back up the necessary files and then connect to a repo for the firmware based on the model number.

 

[Morris]

Why write all this custom code with all the issues that Merlin has mentioned as well as other unknowns? The router already knows how to update it's self so screen scrape and let the router do the work.

 

[L&LD]

For myself, the reason RMerlin firmware is my first/only option with the Asus routers he supports is that I would be in full control of when/if I update it.

This script would never be used. Not being rude here. And this is before RMerlin's posts above.

The resources put towards this project would be better directed elsewhere, if possible.

My 2c.

 

[ExtremeFiretop]

Why write all this custom code with all the issues that Merlin has mentioned as well as other unknowns? The router already knows how to update it's self so screen scrape and let the router do the work.

That's what it currently does now (For the Router Implementation I was starting on). The desktop implementation can't download the firmware directly to the device.

For myself, the reason RMerlin firmware is my first/only option with the Asus routers he supports is that I would be in full control of when/if I update it.

This script would never be used. Not being rude here. And this is before RMerlin's posts above.

The resources put towards this project would be better directed elsewhere, if possible.

My 2c.

Like I said I can drop the router implementation, I just woke up and was thinking of poking some more at the desktop version because I do need it for other things ands flashing firmware is just one of the features which can be skipped.
However if I do decide to use it some day to flash firmware because it notifies me of an "Update Available" in my notification centre, I should probably have at least some of these extra catches in here to secure the update all the way to the final steps.

As to why I've never had a crash in months of upgrading (including all the betas btw) using nothing but hnd-write, I have no idea other than probably hnd-write has it's own set of redundancies and safeties we just aren't aware of?
Is it possible if you initiate a hnd-write remotely it automatically selects the inactive partition every time? It feels not very hard logically to implement.

Again if we don't know for sure the best we could do is decomp, and look for clues as to what ASUS is really doing in their final steps.
I do appreciated the feedback.

 

[L&LD]

Of course, hnd-write may be doing other things behind the scenes, and I agree this is an area to be explored. My hats off to everyone capable of doing so!

As to why you've never had a crash. Luck.

Almost every post here that bypasses Asus' update process (i.e. via the GUI), doesn't have a good outcome for normal users. Not only is wrong firmware possible to upload and brick the router but other anomalies have been reported too.

I collect and use what I believe are 'best practice' solutions and processes for myself and my customers. This is one I would never touch because it is so far into the fringe experimental edge. Even if it worked for the first few thousand/million without any issues, that is no guarantee it will continue to work in the future (nothing is stopping Asus from changing how they do things internally, and they would have no obligation to publicly state any such changes).

First rule of thumb; do no harm, at least, not knowingly. The second rule of thumb, see the first rule.

Generally, IME, anything 'auto' is a false sense of 'everything is working as optimally as it should be'. I'm of the generation that if you want something done right, you do it (manually) yourself.

Not only are you in the mindset of fixing things if/when they go south when you do it manually. But you usually also allocate enough free time to be able to fix things when they don't go smoothly too. And, without it being a state emergency either.

I'll be looking forward to reading your posts and your progress. Just sharing why, it isn't something I would ever use/test. I just don't think this can ever become a bullet-proof option (without Asus' direct involvement/support).

 

[ExtremeFiretop]

Of course, hnd-write may be doing other things behind the scenes, and I agree this is an area to be explored. My hats off to everyone capable of doing so!

As to why you've never had a crash. Luck.

Almost every post here that bypasses Asus' update process (i.e. via the GUI), doesn't have a good outcome for normal users. Not only is wrong firmware possible to upload and brick the router but other anomalies have been reported too.

I collect and use what I believe are 'best practice' solutions and processes for myself and my customers. This is one I would never touch because it is so far into the fringe experimental edge. Even if it worked for the first few thousand/million without any issues, that is no guarantee it will continue to work in the future (nothing is stopping Asus from changing how they do things internally, and they would have no obligation to publicly state any such changes).

First rule of thumb; do no harm, at least, not knowingly. The second rule of thumb, see the first rule.

Generally, IME, anything 'auto' is a false sense of 'everything is working as optimally as it should be'. I'm of the generation that if you want something done right, you do it (manually) yourself.

Not only are you in the mindset of fixing things if/when they go south when you do it manually. But you usually also allocate enough free time to be able to fix things when they don't go smoothly too. And, without it being a state emergency either.

I'll be looking forward to reading your posts and your progress. Just sharing why, it isn't something I would ever use/test. I just don't think this can ever become a bullet-proof option (without Asus' direct involvement/support).

This has been a (generally) very positive and open minded discussion on the topic, thank you to everyone involved, and I'm happy the thread didn't take this way off the rails.

I will definitely update this thread if I encounter issues in continued my testing of the desktop app, or if I make any discoveries along the way (if I do decide to try decomp), but it seems everyone here is pretty much on the same page on their viewpoint of auto-update on the router.
Again I am not fully dropping this, because I am not convinced yet that hnd-write is an automatic dangerous process, assuming the update file makes it to the router okay, it reminds me lots of doing custom firmware flashing on your phone (if you've ever done that at XDA devs)

The partition to flash is always the secondary partition by default, and the reboot at the end is needed at the end to swap to the new partition, since it now has the active flag, on the next reboot that partition is selected as the boot partition and the upgrade is then "complete."
hnd-write does not seem to write directly to my active partition, which obviously would cause a crash instantly as RMerlin mentioned, but if I logged into the Web GUI or connected via SSH AFTER the hnd-write command is complete, then the system in putty and on GUI still report the old version, until I reboot the system.

Now this is just a theory for now, but the evidence for me seems pretty strong that it follows a similar process as flashing firmware updates on a phone in this regard, I haven't had the opportunity yet to dive more into this.
But seems to me that if that really is the situation, and the router itself handles the logic of switching partitions when upgrading (as it should), then the bigger hurdle is already passed and the rest can worked out in stages. (Such as stopping services, unmounting, backing up jffs, etc.)

 

[ExtremeFiretop]

Seems my theory of what I've observed used to be the understanding some years ago: https://www.snbforums.com/threads/a...ble-to-flash-back-the-correct-firmware.72272/

hnd-write allows to write a firmware image, however the same validation process as the webui is applied, so it won't let you cross flash. Which makes me wonder how you managed to flash the wrong model, since the model is validated at flash time when you do so through the webui.

Same here: https://forums.whirlpool.net.au/archive/3w8ynkyy

I believe the newer command hnd-write does checks before flashing to ensure it'll work vs the mtd-write command in the older AC routers.

I'm currently looking for clues in the decomp but without a map file it can be an endless pit:

But I'm almost tempted to just try to flash the wrong firmware on purpose, and see if it takes it or not. (Understanding that a recovery may be needed if it does happen to take the firmware)
But my assumption is it does swap partitions and it also does same basic firmware validation checks. Which is likely why in the 8 months I've been doing upgrades this way. I never crashed or bricked my router.

If it was to flash on the active partition, it would blow away the update file in the update process, since it sits on that same partition, so while I still agree some extra checks can be put into play to help protect the payload all the way until the last step, I don't think I need to specifically be telling the router which partition to flash too and to switch partition flags on the reboot, it handles that logic itself.

 

[ColinTaylor]

@ExtremeFiretop Whilst I applaud your enthusiasm, did you not look at the code RMerlin mentioned in post #27? You seem fixated on only using hnd-write and validating the process on your own router. RMerlin has explained that while that may work fine for you that command is completely wrong for some other models. There may also be additional steps required depending on not only the router model but its hardware revision. And as another poster mentioned, even if you did reverse engineer what the webUI is doing for every model of router there's nothing to stop them changing the process in a future firmware update, as they have done in the past. So the question is not whether it works for you, but whether it can be made to work reliably for 20+ other models.

 

[ExtremeFiretop]

@ExtremeFiretop Whilst I applaud your enthusiasm, did you not look at the code RMerlin mentioned in post #27? You seem fixated on only using hnd-write and validating the process on your own router. RMerlin has explained that while that may work fine for you that command is completely wrong for some other models. There may also be additional steps required depending on not only the router model but its hardware revision. And as another poster mentioned, even if you did reverse engineer what the webUI is doing for every model of router there's nothing to stop them changing the process in a future firmware update, as they have done in the past. So the question is not whether it works for you, but whether it can be made to work reliably for 20+ other models.

Basically any old AC routers that doesn't support hnd-write would need to be unsupported, I would just put a block for those models all together.
So far it's been tested on a GT-AXE11000 and an RT-AX88U and a RT-AX68U without issue for some time, so it's not just about validating it on my router, it's about validating and testing what steps really would need to be scripted and what steps the router handles itself.

 

[Tech9]

it's not just about validating it on my router, it's about validating and testing

It's you and your routers only. No one else will test something that may potentially kill the router.

 

[ExtremeFiretop]

It's you and your routers only. No one else will test something that may potentially kill the router.

That's a lie, I only own the GT-AXE11000, and the RT-AX88U, the AX68U is not owned by me. Either way, the point still stands lol. If hnd-write handles the flash the same way as the webui with similar safety's, the only thing I need to handle the the steps leading up to that.
Any router without hnd-write support would be unsupported, which is why I fixate on how hnd-write functions.

 

[ExtremeFiretop]

There!

The documentation on the Githubs have been updated to reflect the unsupported models, and supported models, and the scripts have been adjusted to quit without action if those models are detected incase someone decides to try to ignore the warnings and documentation.

Let me know if anyone notices any mistakes in the models identified This again was a good catch and good discovery by dave14305 and RMerlin, but not the end of the road for all the supported model, so far anyways...