RT-AX86U – AiProtection + QNAP HybridSync Mount Failures After Updating to 3004.388.10 (Resolved by Downgrade to 388.9)

[themilfalcon]

Hi all,
Posting this in case others run into the same issue, and to understand if there’s a known cause.
Router: RT-AX86U (Zaku II Edition)
Previous stable: 3004.388.9 (Merlin)
Updated to: 3004.388.10 (Merlin)
NAS Setup: Two QNAP units (latest QTS), syncing daily using Hybrid Backup Sync (mount-based file replication, not rsync push)

---

Issue After Updating to 3004.388.10

Immediately after upgrading to 3004.388.10, my scheduled QNAP → QNAP HybridSync jobs started failing intermittently:

• Remote NAS hostname resolved normally
• Mounts would either fail outright, or mount but stall partway through file operations
• Manual sync attempts from QNAP UI also failed

No network, DNS, or NAS configuration changes were made — the only change was the router firmware update.

---

Additional Finding:

Although the web UI showed AiProtection enabled, the Trend Micro DPI components were not running:

• DPI processes were absent in ps
• No intrusion/web filter logs were appearing
• Restarting the firewall or toggling AiProtection did not start DPI services

This suggests that DPI / firewall module load order or initialization may have been affected in 3004.388.10.
Since QNAP mount and file sync flows pass through the DPI chain, this could explain the mount instability and stalled sync transfers.
Tried a hard reset and also tried a reset by clearing the NVRAM. The problem did not go away.

---

Resolution
I downgraded back to 3004.388.9:

• AiProtection / DPI processes immediately resumed normal operation
• QNAP HybridSync mounts and transfers now run fully stable, as before

---

Request to Community / Developers

• Has anyone else seen AiProtection not actually running under 3004.388.10?
• Is there a known change to DPI, netfilter hooks, or service startup order in this release?
• If needed, I can capture logs before/after.

Posting this in case it helps others diagnosing similar symptoms.
Thanks to everyone contributing to Merlin — it has been rock solid for my environment outside of this one case.

 

[ColinTaylor]

Can you clarify:

This is a backup (or sync) between two QNAP servers. Are these servers on the same LAN, or are they in different locations and you're doing this over the internet?

As this is a Merlin sub-forum can you briefly explain what underlying protocol this "HybridSync" uses. You said it isn't rsync.

 

[themilfalcon]

1. Network Topology


Both QNAP units are on the same LAN, same subnet:

QNAP #1 → RT-AX86U → QNAP #2

No VPN, no WAN routing involved.
This is strictly LAN-side, internal traffic.

2. Hybrid Backup Sync Protocol Used
Hybrid Backup Sync (HBS3) supports multiple transfer modes (rsync, RTRR, S3, etc).


However, in this case, the job is configured as a mount-based sync, meaning:

• QNAP #1 mounts a shared folder on QNAP #2 directly using SMB (CIFS)
• Then performs a local file-level comparison + copy, not rsync push.

To emphasize:
This is not rsync and not RTRR over WAN.
It’s SMB over LAN, initiated by one QNAP mounting the other like a local network share.

Thanks

 

[ColinTaylor]

Thanks for the info.

Given this is LAN to LAN traffic I can't think why the router's firmware would have any bearing. The only time the NAS traffic hits the router is when it's switched between the LAN ports (assuming both are ethernet connected). None of the traffic is being routed so things like AiProtection, DPI, firewall, QoS, etc. don't come into play.

Does the NAS have any logging that might give a clue?

I suppose it's possible that the router was having major problems more generally (e.g. reboot loop, processes crashing or CPU at 100%), and that the NAS issues were just a side effect of that. Any clues in the router's System Log - General Log?

 

[bennor]

@themilfalcon, as ColinTaylor suggested, check the logs on both the router and both NAS's to see what information they might shed as to what's happening around the time the NAS's are performing a backup.

What other troubleshooting steps have you tried?
Have you tried restarting the router and both NAS's?
May have missed it, but are you running any addon scripts on the router or have any custom configurations added via SSH to the router?
Disable the router's AiProtection AND withdraw from it on the Administration > Privacy (or Policy) page as a troubleshooting step.
Does the NAS to NAS backup fail randomly or at a specific location/time in the backup?

Also, if possible can you clean up your posts a bit to remove the extra blank lines? It makes the post(s), particularly your first post, flow like a wall-o-text with multiple blank lines between sentences, paragraphs, and bulleted list blocks.

 

[themilfalcon]

No

@themilfalcon, as ColinTaylor suggested, check the logs on both the router and both NAS's to see what information they might shed as to what's happening around the time the NAS's are performing a backup.

What other troubleshooting steps have you tried?
Have you tried restarting the router and both NAS's?
May have missed it, but are you running any addon scripts on the router or have any custom configurations added via SSH to the router?
Disable the router's AiProtection AND withdraw from it on the Administration > Privacy (or Policy) page as a troubleshooting step.
Does the NAS to NAS backup fail randomly or at a specific location/time in the backup?

Also, if possible can you clean up your posts a bit to remove the extra blank lines? It makes the post(s), particularly your first post, flow like a wall-o-text with multiple blank lines between sentences, paragraphs, and bulleted list blocks.

No add on scripts. Tried restarting everything many times. no custom configuration. The mountsync was working until i upgraded the firmware and started working after downgrade.
Logs belwo
PS:cleaned up the original post. I wrote it on word and then copied it here so many empty lines appeared.

Router Log Symptoms Observed on 3004.388.10​

1. CIFS/SMB Mount Handshake Timeouts
kernel: CIFS VFS: BAD_NETWORK_NAME: \\<QNAP2-IP>\<sharename>
kernel: CIFS VFS: cifs_mount failed w/return code = -112
kernel: CIFS VFS: unable to reconnect: errno = -5
2. Session Establishment Dropping / Stalling
kernel: net_ratelimit: 12 callbacks suppressed
kernel: tcp: request_sock_TCP: Possible SYN flooding on port 445. Check SNMP counters.
This indicates SMB handshake attempts stalled or were dropped.
**3. (No Trend Micro service startup entries, which normally appear at boot)
What we saw on 388.10:
rc_service: restart_firewall
rc_service: service start firewall
# (No DPI service initialization lines followed)

--On 388.9 these services are started.
rc_service: dnsfilter 37:notify_rc restart_firewall
tm_monitor: DPI engine started
tm_fwd: initializing traffic inspection hooks

4. Kernel Hook Warnings (likely side effect)
kernel: nf_conntrack: table full, dropping packet
kernel: netfilter: nf_conntrack_dpi: module not loaded

During the issue, the router syslog consistently showed CIFS mount retries and handshake failures:
kernel: CIFS VFS: cifs_mount failed w/return code = -112
kernel: CIFS VFS: unable to reconnect: errno = -5
kernel: tcp: request_sock_TCP: Possible SYN flooding on port 445.

At the same time, AiProtection DPI daemons were not running, and the normal DPI startup log entries did not appear after boot.
This suggests DPI / netfilter initialization failure in 3004.388.10, which in turn disrupted stable SMB mounts on LAN.

hope this helps.

 

[ColinTaylor]

No add on scripts. Tried restarting everything many times. no custom configuration. The mountsync was working until i upgraded the firmware and started working after downgrade.

I don't see why the router would be trying to mount shares from your NAS. Your NAS to NAS backup should not be touching the router at all.

Are you sure you're not running a custom script like BACKUPMON. The router normally has no ability to perform this sort of mount (unless perhaps you've enabled AiCloud - not sure about that).

During the issue, the router syslog consistently showed CIFS mount retries and handshake failures:
kernel: CIFS VFS: cifs_mount failed w/return code = -112
kernel: CIFS VFS: unable to reconnect: errno = -5
kernel: tcp: request_sock_TCP: Possible SYN flooding on port 445.

Are you sure these messages are from the router's syslog and not the QNAP's log?

 

[bennor]

The mountsync was working until i upgraded the firmware and started working after downgrade.

Router Log Symptoms Observed on 3004.388.10​

1. CIFS/SMB Mount Handshake Timeouts

Why is your router trying to mount a share on the NAS(s)?
How exactly have you configured the NAS units to back up from one to the other? Have you setup some sort of script on the router to perform the backup from one NAS to the other NAS?
If you have a network switch, as a troubleshooting step, connect the two NAS units to that Network switch then connect the switch to the router and see if the NAS backup works.

 

[themilfalcon]

Thanks for the questions — let me clarify the setup:
1. The router is not mounting anything.
The mount operation is initiated by one QNAP NAS, not the router.
The QNAP Hybrid Backup Sync job is configured as:
NAS #1 → mount SMB share from → NAS #2
So the backup is NAS-to-NAS.
The router’s role is simply to route LAN traffic, nothing special configured on it.
There are no scripts on the router, no cron jobs, no rsync jobs running on the router itself.

2. Configuration in HBS3 on QNAP
Hybrid Backup Sync (HBS3) is set to:

• Source: Local Shared Folder on NAS #1
• Destination: Remote NAS shared folder on NAS #2
• Transfer Method: SMB / CIFS mount, not rsync and not RTRR

This means NAS #1 issues a standard SMB mount to NAS #2 and performs file-level copy operations locally.
So the data path is simply:
NAS #1 → Router LAN Switch → NAS #2
No WAN, no VPN, no routing complexity.

3. Regarding your switch suggestion
Both NAS units are already connected through the router’s built-in LAN switch backplane (same L2 segment).
However, your suggestion is good for validation — I did test with a standalone unmanaged switch in between, and the same issue occurred on 3004.388.10, but immediately disappeared when downgrading back to 3004.388.9.
So the behavior is firmware-dependent, not physical-topology-dependent.

Key observation​

On 3004.388.10, AiProtection (Trend Micro DPI) processes were not running, even though the WebUI showed it enabled.
Without DPI hooks active, SMB session negotiation and sustained file transfers were intermittently failing.
After downgrading to 3004.388.9:

• DPI services started normally
• SMB mounts became stable again
• QNAP sync jobs returned to normal reliability

I also remember a similar issue last year and I had to downgrade. I am not sure if it was 388.7. or 388.8.
“I reviewed the Merlin 388.x changelog. While 3004.388.8 lists a fix for CIFS mount failures on BCM4912 devices, I found no mention of DPI/AiProtection daemons not loading, nor of LAN NAS-to-NAS sync mounts failing in 3004.388.10. This indicates this issue may be unreported in the public release notes for my router model (RT-AX86U Zaku II).”
hope this helps.

Why is your router trying to mount a share on the NAS(s)?
How exactly have you configured the NAS units to back up from one to the other? Have you setup some sort of script on the router to perform the backup from one NAS to the other NAS?
If you have a network switch, as a troubleshooting step, connect the two NAS units to that Network switch then connect the switch to the router and see if the NAS backup works.

 

[ColinTaylor]

Thanks for the clarification @themilfalcon. But that still doesn't explain why your router is trying to mount a CIFS filesystem.

In fact most of the log messages you've posted aren't present in the firmware at all. So either you've mistyped them or they're from a different system.

I think the only way we're going to be able to understand what is happening is for you to recreate the problem by reinstalling 388.10 and then post the complete syslog (including the timestamps) from the moment the router boots up.

N.B. Go to System Log - General Log and set Log only messages more urgent than to "all".

 

[themilfalcon]

Thanks for the clarification @themilfalcon. But that still doesn't explain why your router is trying to mount a CIFS filesystem.

In fact most of the log messages you've posted aren't present in the firmware at all. So either you've mistyped them or they're from a different system.

I think the only way we're going to be able to understand what is happening is for you to recreate the problem by reinstalling 388.10 and then post the complete syslog (including the timestamps) from the moment the router boots up.

N.B. Go to System Log - General Log and set Log only messages more urgent than to "all".

OK will try to update again and check.
I also just found purely by accident. The NAS was connected to the router using 802.ad one of the links was downgraded to 100mb/s and the sync errors happened. I have changed the cable now. Let me see if the problem goes away for good.