XT8 5GHz-2 Backhaul Doesn't Hold - Any Final Suggestions?

[slug56]

Hello,

I've had a pair of XT8s for the last few months, configured with a wireless (5GHz-2) backhaul. Every few hours/days, the backhaul drops to 2.4GHz, interrupting any clients connected to the secondary node and severely reducing the throughput. I've reached a point where I think I've tried everything I can think of and it looks like ASUS support has also given up on my case now. Before I sort out an exchange/refund from the retailer, I was wondering if anyone had any final thoughts?

Observations:

• I get lots of these messages in my node's dmesg every hour when the WPA2 key is rotated. Not sure if they're normal.

  CFG80211-ERROR) wl_cfg80211_set_rekey_data : seting gtk_key_info failed code=-23

• Whenever the connection drops, I can't see anything noteworthy in the main router's logs/dmesg.
• Whenever the connection drops, I see the following in the secondary node's logs:

  Mar 23 14:13:28 wpa_supplicant[1958]: eth6: CTRL-EVENT-DISCONNECTED bssid=ROUTER_5GHZ_2_BACKHAUL_MAC reason=4
  Mar 23 14:13:30 BHC: WiFi connection status change.
  Mar 23 14:13:30 BHC: bandindex(0): state is 2
  Mar 23 14:13:30 BHC: bandindex(1): state is 0
  Mar 23 14:13:30 BHC: bandindex(2): state is 0
  Mar 23 14:13:30 BHC: Topology change from 8 to 2.
  Mar 23 14:13:39 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 0
  Mar 23 14:13:41 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ disconnect
  Mar 23 14:13:41 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 1
  Mar 23 14:13:41 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ reconnect

• Whenever the connection drops, I see the following in the secondary node's dmesg:
  

  CFG80211-ERROR) wl_cfg80211_set_rekey_data : seting gtk_key_info failed code=-23
  CFG80211-ERROR) wl_notify_connect_status : link down if eth6 may call cfg80211_disconnected. event : 6, reason=4 from ROUTER_5GHZ_2_BACKHAUL_MAC
  device dpsta.502 left promiscuous mode
  br2: port 2(dpsta.502) entered disabled state
  device dpsta.502 entered promiscuous mode
  br2: port 2(dpsta.502) entered listening state
  br2: port 2(dpsta.502) entered listening state
  br2: port 2(dpsta.502) received tcn bpdu
  br2: topology change detected, propagating
  cfg80211: Exceeded CRDA call max attempts. Not calling CRDA
  br2: port 2(dpsta.502) entered learning state
  br2: topology change detected, propagating
  br2: port 2(dpsta.502) entered forwarding state

• Once the backhaul has dropped to 2.4GHz, the secondary node's logs get spammed with the following until it is reset:
  

  Mar 23 15:46:05 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 0
  Mar 23 15:46:07 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ disconnect
  Mar 23 15:46:12 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 0
  Mar 23 15:46:14 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ disconnect
  Mar 23 15:46:14 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 1
  Mar 23 15:46:14 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ reconnect
  Mar 23 15:49:17 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 0
  Mar 23 15:49:19 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ disconnect
  Mar 23 15:49:24 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 0
  Mar 23 15:49:26 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ disconnect
  Mar 23 15:49:26 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 1
  Mar 23 15:49:26 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ reconnect
  Mar 23 15:52:28 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 0
  Mar 23 15:52:30 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ disconnect
  Mar 23 15:52:35 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 0
  Mar 23 15:52:37 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ disconnect
  Mar 23 15:52:37 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 1
  Mar 23 15:52:37 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ reconnect
  Mar 23 15:55:39 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 0
  Mar 23 15:55:41 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ disconnect
  Mar 23 15:55:46 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 0
  Mar 23 15:55:48 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ disconnect
  Mar 23 15:55:48 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ sta_autoconnect 1
  Mar 23 15:55:49 PtyConn: [SUCCESS] wpa_cli-2.7 -p /var/run/wl2_wpa_supplicant/ reconnect

I have tried the following to try and remediate the issue:

• Many (many, many) hard resets using WPS button and not restoring from settings files
• Fixing at various channels and bandwidths for all 3 radios
• Disabling roaming assistant
• Disabling smart connect
• Changing WPA2 passkeys
• Changing group rotation intervals to be greater than 3600 seconds
• Setting a "Preferred WiFi Uplink AP"
• Changing the Backhaul Connection Priority to "5GHz-2 WiFi first"
• Physically swapping the main router and secondary node
• Removing my one 5GHz-1 guest network
• Trying WPA3-Personal instead of WPA2-Personal
• Disabling universal beamforming

I have/am:

• Running 3.0.0.4.388_22525 firmware
• Never seen any suggestion in the Wireless Log or otherwise that interference has been detected
• Currently using 2.4GHz channel 1 (20 MHz), 5GHz-1 channel 52 (80 Mhz) and 5GHz-2 channel 100 (80 Mhz)
• In the UK

Any further ideas?

 

[markpmac]

I used to have the same exact issue. The only thing that worked for me out of all the remedies you'll see posted was switching to GNUton's Merlin releases. I'm currently so rock solid on 386.07 that I'm not even planning to upgrade to 388.1.

 

[slug56]

I used to have the same exact issue. The only thing that worked for me out of all the remedies you'll see posted was switching to GNUton's Merlin releases. I'm currently so rock solid on 386.07 that I'm not even planning to upgrade to 388.1.

Ah, thanks.

I tried GNUton Merlin - albeit 388.1, but the backhaul has just died again and reverted to 2.4GHz with pretty much the same error

Mar 27 13:00:40 wpa_supplicant[1963]: eth6: CTRL-EVENT-DISCONNECTED bssid=ROUTER_5GHZ_2_BACKHAUL_MAC reason=4
Mar 27 13:00:40 kernel: CFG80211-ERROR) wl_notify_connect_status : link down if eth6 may call cfg80211_disconnected. event : 6, reason=4 from ROUTER_5GHZ_2_BACKHAUL_MAC

Thanks for the suggestion.

 

[degrub]

try running an ethernet cable temporarily to the node and see if that helps the issue.

 

[slug56]

try running an ethernet cable temporarily to the node and see if that helps the issue.

That's my long-term goal, but not something I can practically do at the moment.

I've started a return of my units and have some replacements on the way, so that'll hopefully tell me if I have a hardware problem or perhaps if there's something environmental at play.

 

[slug56]

Got some replacement units today. Set them up from scratch on the latest firmware. Not 30 minutes later, the backhaul has dropped to the 2.4GHz radio in exactly the same manner as the original pair.

I can't understand it it's got to be something environmental, hasn't it? It coincidentally almost always happens when streaming from an LG webOS TV, whatever difference that makes.

 

[j4y-t]

I think channel 100 is a DFS channel, which is susceptible to interruption if it detects any RADAR signals. Can you change it to a non-DFS channel (e.g., I'm using 149, although I'm in Canada)?

 

[joe_]

I've been dealing with this same issue daily since... I dunno, 2020? I tried everything imaginable and it still won't take. Funny enough; I had almost a whole year since late 2021 where it was working perfectly without an drops. Literally. I never even had to restart the nodes ever. This lasted forever until i had a sudden electrical outage. Upon turning the routers back on; the issue came back and I never saw a long period of time where it was stable like that other than a handful of weeks or so.

I recently decided to flash merlin into the nodes (following all the proper steps, factory resets, etc), and it seemed to have fixed it........ all for three days until it started happening again. I also noticed that it's happening even more frequently every day than it used to.

I'm just gonna keep dealing with this BS a while longer till I change my whole set up and get a new router installed. Will then proceed to take a hammer to this garbage mesh system and smash both nodes to bits.

One of the worst investments I ever made.

 

[JackHennety]

Nothing to contribute other than sympathy @joe_ . It's the unpredictability I can't stand. After months of problems with a node dropping back to 2.4GHz, and constant factory resets and starts from scratch, Merlin, you name it, I thought everything was going ok with 2 whole months of solid 5GHz. Then at the weekend, boom, my second node drops back to 2.4Ghz just like old times. No amount of hard resets is making a difference now. Connection is 'Great' to start with, and within minutes it's 'Weak'. I hate this thing.