SCP & RSYNC Constantly Break

[GameDr04]

I have been having a very bizarre issue with terminal-based file transfers for the past couple months, now.
This issue has taken a full five weeks of the most outrageous troubleshooting steps to identify (it's how I roll).

I removed the cable that Xfinity put on the outside of my house and replaced it with my own, full-copper, quad-shield cable that I buried in conduit in my backyard, myself.
I upgraded and changed my modem...four times (by the way, Motorola MB8611 and Netgear CM2050V are pretty great.)

After all that...it seems to be at the router level.

Any time I scp or rsync something larger than a few megabytes from home to one of my many off-premises servers, I will inevitably be presented with this error:

client_loop: send disconnect: Broken pipe

rsync: [sender] write error: Broken pipe (32)

HOWEVER
If I connect to a VPN on the sending machine at home and scp/rsync through that, every transfer is guaranteed to complete successfully.
This is true for NordVPN, three of my friends' OpenVPN servers, and even a PPTP VPN I setup at work just to test this.

A few years ago, somebody mentioned a similar problem on SNB, and he narrowed it down to CTF (NAT Acceleration).

https://www.snbforums.com/threads/ssh-broken-pipe-somehow-related-to-nat-acceleration.50294/

Unfortunately, disabling CTF is not an option on the AX11000 so I can't readily test that.

Has anybody else been having problems with SSH, SCP, RSYNC, or any other such terminal-level things?

 

[eibgrad]

Usually if you enable something that's incompatible w/ CTF, the router will *silently* disable it, for example QoS or any kind of bandwidth limiting.

It may be an MTU issue as well.

 

[Tech Junky]

Sounds to me like a CC issue since when you wrap the traffic in the VPN it works fine.

BTW I use Nord as well w/ WG enabled for faster speeds now for ~4 years w/ very few issues and put it on the "router" for LAN protection regardless of the device. Everything is tunneled.

 

[Justinh]

After all that 'outrageous' stuff... Do you have another router you can test on?

Instead of a router issue, could it be Xfinity?
DNS vs IP address?
What is diff between VPN and !VPN?

 

[GameDr04]

Sounds to me like a CC issue since when you wrap the traffic in the VPN it works fine.

I apologize, but I don't know what CC means in this context.

Usually if you enable something that's incompatible w/ CTF, the router will *silently* disable it, for example QoS or any kind of bandwidth limiting.

Thank you for this! I enabled "Traditional QoS" and started my big (20GB) rsync test. However, I stopped it manually because normal Internet tasks were damn-slow. I'll try it overnight and get back to you.

Do you have another router you can test on?

I do. I tested it on my previous router, an RT-AC5300, then the one before that which is an RT-AC68U. All three routers are running Merlin's 386.7, but this was definitely a problem with 386.5_2 and possibly with 386.5_0.
All three fail at different times during the rsync process.
None of them has anything in the system log corresponding in time with the failures.

Instead of a router issue, could it be Xfinity?

My sister lives just down the street and is also on Xfinity. I set her up with a GT-AC5300 and I ran the same 20GB rsync without a VPN. Hers succeeds every time, independent of the destination.
That doesn't exactly rule out Xfinity, but it helps.
What it also does is cast some suspicion on Stock and/or Merlin Firmware (definitely not disparaging here, I buy routers with Merlin in mind).

What is diff between VPN and !VPN?

!VPN: >99% chance of failure for transfers greater than a few hundred megabytes
VPN: transfer is upwards of 25% slower; guaranteed success


I've had this AX11000 since mid-December and the problem only started happening a couple months ago.

Later today, I plan to downgrade my firmware to see if it helps.

I have a temporary, server-side solution to this but it's exceedingly inconvenient. More to the point: it doesn't solve this delicious mystery.

 

[Tech Junky]

I apologize, but I don't know what CC means in this context.

ComCast

 

[Justinh]

Right. So what happened in December?? Always the first place to look.

The VPN vs !VPN question was not about results (you gave those) but with what the system is doing, what piece of the network puzzle changes.