I recently took some AC5300's I had and repurposed them for my wife's cousin. One as a router, and the other as a node. I could not add the second 5300 nor could it be found to add it as a node.
My goal was to add using wireless, then move to a wired back haul. Both were on Merlin 386.3 and I upgraded both to Merlin 386.10 beta before deploying (need to run a couple scripts on the node, hence using Merlin, not stock, on it. During the upgrade, I did something to the config for that node 5300 that prevented being able to add it. But I did eventually add it, over a wired backhaul.
What worked for me, before adding the mesh node I set the router to prefer Ethernet backhaul, first. Then connected WAN on node 5300 to a LAN port on router 5300 (note: still wouldn't find/add it). Then I WPS reset the node 5300 while connected, and within seconds after after rhe reboot and the WAN became active on that node 5300 (based on the 5300 WAN inticator LED coming on) I went back to add an AIMesh node. Immediately was discovered and I was able to add the second 5300 as a node. One the router 5300 WPS was enabled not sure if that really matters anyway, the node was fresh off a WPS reset, hadn't even logged into it.
The only thing I can come up with is that logging into the node after the initial WPS reset under 386.3 to upgrade the firmware the first time set/changed something that messed with the discovery process, even setting it's operational mode to AiMesh node didn't help. I had to set the preference for Ethernet backhaul on the router, then get the physical connectivity done, then both router and node powered up. After which a WPS reset on the node (clearing out ID/password on the node is what, I suspect, allowed the device discovery to occur) to start it up clean/default as it came back up, only then was it discovered allowing me to add it.
Then after both were up, while configuring the 2.4Ghz wireless to turn off 802.11b on the router, I then had to SSH into the node to set nvram to the same seting (to set in nvram to disable 802.11b is wl0_rateset=ofdm) on the node, commit it and reboot. Otherwise while the router has 802.11b disabled, the node(s) has it enabled slowing things down on the 2.4Ghz WiFi. That setting isn't propagating from the router, an AiMesh/Wireless closed source issue not Merlin specific.
Unfortunately I have not been able to go back to repeat the process and test adding a node in a measured/controlled/repeatable way, sorry...
(Teen age daughter on her iPhone/iPad home on Spring Break, on the devices all day & night, online