Have you downloaded the proper file for the router you actually have? Firmware is not interchangeable.
Have you verified the hash of the extracted download?
Do you have any USB devices attached? If so, Safely remove them via the GUI, then physically remove them. Reboot, and attempt the installation once more.
Also, simply reboot via the GUI, and try to install once more (whether you have USB devices attached or not).
If that doesn't work, I would disconnect everything from the router (WAN, LAN, USB, etc.), perform an electrical reset, power it up, and then attempt to install it once more.
If all the above doesn't work, then perform the last step above and perform a full reset on the router (see link below). Do not connect anything (even WAN), just do a minimal config to be able to install the firmware you want to use. At a successful install, you need to do another full reset after successfully moving from stock to RMerllin firmware (or, vice-versa).
https://www.asus.com/us/support/FAQ/1039074
After successfully updating and fully resetting the router on RMerlin firmware, do the same to the XT8s before attempting to add them as AiMesh nodes too.
But first, to quickly get back to where you are now, follow the steps outlined in the link below.
I have had issues in the past when updating firmware and Internet speed, But it may have just been a fluke, Should I keep up with the updates? If so On all routers or the nodes as well? What about ones acting as access points? Thanks
www.snbforums.com