That is weird. @mscrivo had the same error message (see https://www.snbforums.com/threads/rp-ax56-cant-use-ethernet-backhaul.70856), whereas it's working fine with my XT8 as the router.
I haven't done any scientific tests, but I have walked around the house with a wifi analyzer app looking at the 5GHz band only. I conclude that the RP-AX56 seems to have better range compared with the 5GHz-1 radio of the XT8, but slightly worse range compared with the 5GHz-2 radio of the XT8.
I am using the RP-AX56 as an aimesh node, connected using ethernet backhaul. See: https://www.snbforums.com/threads/rp-ax56-cant-use-ethernet-backhaul.70856/#post-673401
Encountering the same thing on my XT8 based mesh nnetwork. Guest network 1 will not work unless it is given intranet access. Solution is to use guest network 2. But I don't want to use guest network 2, because only guest network 1 is propagated across all nodes. Ridiculous!
So I decided to buy this repeater anyway to test it out, and interestingly ethernet backhaul works for me! I'm using an XT8 running the latest stock firmware (3.0.0.4.386_41793) as the router and an RP-AC56 running the only firmware available (3.0.0.4.386_26131)
Registered to ask whether you resolved this. I was hoping to get this repeater for the same purpose. Funny thing is, the manual for it, straight from Asus, says that is should support ethernet backhaul: https://dlcdnets.asus.com/pub/ASUS/wireless/RP-AX56/E16563_RP-AX56_V1_QSG.PDF