multisecond ping latencies on wireless

I have intermittent problems with multisecond RTT latencies on my wifi network. It seems like packets get stuck for multiple seconds and then after a while show up. It's not only pings, other traffic as well. My ssh sessions from a wireless client to a wired machine (Ethernet to the router/AP) also hang during these "occurrences".

Pinging from a Dell XPS 13 running Linux Mint to my Archer C7 V2 router:
64 bytes from 192.168.1.1: icmp_seq=303 ttl=64 time=9575 ms
64 bytes from 192.168.1.1: icmp_seq=304 ttl=64 time=8576 ms
64 bytes from 192.168.1.1: icmp_seq=305 ttl=64 time=7576 ms
64 bytes from 192.168.1.1: icmp_seq=306 ttl=64 time=6576 ms
64 bytes from 192.168.1.1: icmp_seq=307 ttl=64 time=5576 ms
64 bytes from 192.168.1.1: icmp_seq=309 ttl=64 time=3576 ms
64 bytes from 192.168.1.1: icmp_seq=310 ttl=64 time=2576 ms
64 bytes from 192.168.1.1: icmp_seq=311 ttl=64 time=1576 ms
64 bytes from 192.168.1.1: icmp_seq=312 ttl=64 time=594 ms
64 bytes from 192.168.1.1: icmp_seq=313 ttl=64 time=1.63 ms
64 bytes from 192.168.1.1: icmp_seq=314 ttl=64 time=2.02 ms
See the pattern? Ping transmits a packet every second. 9 of them get stuck somewhere. Then all of them (bar one in this case, #308 is missing) are suddenly "released" and received by the client. After that I get the expected latency. Sometimes the pings (requests or replies) are also lost.

I originally had a TP-LINK TL-WR1043ND V1.8 (w. latest firmware). It showed the same problem (pinging from a Samsung ChromeBook 2 (ARM) running ChromeOS). I thought it was a software problem so installed OpenWRT on the 1043ND. The problem did not go away. OK maybe it is a hardware problem. Thus I bought the Archer C7 V2 (one of the few 802.11ac router/AP with OpenWRT support, I liked OpenWRT and plan to install it later). Upgraded to the latest stock firmware. Pinging the AP from the XPS13 machine. Same problem.

So I have tested with different router HW and SW and with different clients but the problem persists. It seems to happen (more frequently or only?) and with worse latency when there are other heavy wifi users (e.g. someone streaming youtube to a tablet, this happens a lot). I can understand that that should cause more latency and jitter, I sometimes see that type of behaviour from ping (intermittent random latencies up to 200ms). But not the above very regular behaviour.

It happened again! This time I was pinging a machine (192.168.1.2) connected to the router (192.168.1.1) using Ethernet (wireless client -> AP ->(Ethernet) 192.168.1.2 and back) so the router is just forwarding the packets between wireless and wired networks. I assume this is done in SW and not automatically by the switch in the router.

olli@xps13 ~ $ iwconfig
lo no wireless extensions.

wlan0 IEEE 802.11abgn ESSID:"olli"
Mode:Managed Frequency:2.437 GHz Access Point: XXXXXXXXXXXX
Bit Rate=39 Mb/s Tx-Power=16 dBm
Retry long limit:7 RTS thrff Fragment thrff
Power Managementn
Link Quality=40/70 Signal level=-70 dBm
Rx invalid nwid:0 Rx invalid crypt:0 Rx invalid frag:0
Tx excessive retries:0 Invalid misc:310 Missed beacon:0

olli@xps13 ~ $ ping 192.168.1.2
PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data.
64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=4036 ms
64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=3029 ms
64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=2024 ms
64 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=1018 ms
64 bytes from 192.168.1.2: icmp_seq=5 ttl=64 time=13.2 ms
64 bytes from 192.168.1.2: icmp_seq=6 ttl=64 time=3.77 ms
^C
--- 192.168.1.2 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5031ms
rtt min/avg/max/mdev = 3.777/1687.749/4036.380/1501.003 ms, pipe 5
olli@xps13 ~ $ iwconfig
lo no wireless extensions.

wlan0 IEEE 802.11abgn ESSID:"olli"
Mode:Managed Frequency:2.437 GHz Access Point: XXXXXXXXXXXX
Bit Rate=52 Mb/s Tx-Power=16 dBm
Retry long limit:7 RTS thrff Fragment thrff
Power Managementn
Link Quality=41/70 Signal level=-69 dBm
Rx invalid nwid:0 Rx invalid crypt:0 Rx invalid frag:0
Tx excessive retries:0 Invalid misc:310 Missed beacon:0

See anything strange?

Possibilities
1. heavy RF interference from a non-WiFi source (assuming no hardware faults). Move to a different channel among 1, 6 or 11
2. Router firmware is bogged down due to some firmware flaw, a fould up ARP table (reboot to see), substitute different router
3. Faulty transmissions from client. Does this happen on all clients?

> 1. heavy RF interference from a non-WiFi source (assuming no hardware faults). Move to a different channel among 1, 6 or 11
I doubt RF interference would create such deterministic latencies. Perhaps if all outgoing packets were stuck on the client egress side due to the transmitter not being able to gain access to the "ether" and then after N seconds, all enqueued packets are transmitted at once. I will try to hardcode different channels, I think the selection is automatic now.

> 2. Router firmware is bogged down due to some firmware flaw, a fould up ARP table (reboot to see), substitute different router
Tried different routers so should be different firmware. Both of the routers use Qualcomm/Atheros chipsets (Atheros AR9103 2.4 GHz 802.11bgn resp. QCA9558 3x3 b/g/n per OpenWRT web site) though so there is a potential for SW reuse and thus shared bugs here.

> 3. Faulty transmissions from client. Does this happen on all clients?
I have tried two different clients. Both Linux-based but I doubt the problem lies there. I will try with some other computer, I can dig up something with Windows.

re (1).. might be some 2.4GHz thing like a baby monitor or wireless security camera in 2.4GHz. those consume about 6MHz so moving to the opposite end of the band is recommended. But you need to disable automatic frequency selection.

Disable all power-saving features at the wireless client.

Your error rate seems low, so that is good. Seems to rule out interference.

You might connect to the router via telnet/SSH and see if you can get any details (errors, interference, etc) from the "wl" command.

Nullity said:

Disable all power-saving features at the wireless client.

Your error rate seems low, so that is good. Seems to rule out interference.

You might connect to the router via telnet/SSH and see if you can get any details (errors, interference, etc) from the "wl" command.

Click to expand...

But.. .the OP says the incorrect ping times occurred with different router hardware and different clients.
What's left? He's yet to say he changed channels and turned off auto-channel-selection.

Did the same AC wall transformer get used for both routers?

stevech said:

But.. .the OP says the incorrect ping times occurred with different router hardware and different clients.
What's left? He's yet to say he changed channels and turned off auto-channel-selection.

Did the same AC wall transformer get used for both routers?

Click to expand...

Ah, good point. I missed that this happened with two different routers. Though, the first router was tested with a Chromebook which I assume has sub-par "mobile" wireless hardware. My Nexus 7 (2012) has a fluctuating ping of 1-20ms with an occasional 130ms response but all of my Desktop/laptop wireless clients have sub-millisecond pings.


Perhaps your LAN has a faulty NIC/cable/switch somewhere?

Thanks for the suggestions. I'll connect to the router and try to get some statistics out of it.

Since the new router (Archer C7) supports both 2.4GHz and 5GHz bands, I switched to using the 5GHz band for the XPS13 machine. The XPS13 doesn't support 802.11ac so I think it using 802.11n. Ping latencies (to the router, not to the wired host) looks much better now although I still get infrequent latencies in the hundred of milliseconds (and often 2-3 pings after each other so some level of temporal correlation), see below. But this pattern of N pings getting stuck and then released at once is no more. So some type of RF interference seems likely. I will try 2.4GHz again but with different channels.

olli@xps13 ~ $ ping 192.168.1.1 | grep '.*time=[0-9][0-9][0-9] ms'
64 bytes from 192.168.1.1: icmp_seq=44 ttl=64 time=272 ms
64 bytes from 192.168.1.1: icmp_seq=45 ttl=64 time=142 ms
64 bytes from 192.168.1.1: icmp_seq=162 ttl=64 time=171 ms
64 bytes from 192.168.1.1: icmp_seq=163 ttl=64 time=119 ms
64 bytes from 192.168.1.1: icmp_seq=166 ttl=64 time=167 ms
64 bytes from 192.168.1.1: icmp_seq=174 ttl=64 time=418 ms
64 bytes from 192.168.1.1: icmp_seq=226 ttl=64 time=348 ms
64 bytes from 192.168.1.1: icmp_seq=281 ttl=64 time=234 ms
64 bytes from 192.168.1.1: icmp_seq=283 ttl=64 time=233 ms
64 bytes from 192.168.1.1: icmp_seq=284 ttl=64 time=129 ms
64 bytes from 192.168.1.1: icmp_seq=404 ttl=64 time=207 ms
64 bytes from 192.168.1.1: icmp_seq=405 ttl=64 time=128 ms
64 bytes from 192.168.1.1: icmp_seq=520 ttl=64 time=278 ms
64 bytes from 192.168.1.1: icmp_seq=523 ttl=64 time=100 ms
64 bytes from 192.168.1.1: icmp_seq=525 ttl=64 time=225 ms
64 bytes from 192.168.1.1: icmp_seq=526 ttl=64 time=146 ms
64 bytes from 192.168.1.1: icmp_seq=528 ttl=64 time=394 ms

This is at a different time of day (morning, not even evening which is when I usually do my tests) so external conditions may be different, I am also only 1m away from the router. I will continue testing tonight.

The different routers use different AC transformers, whatever transformer was delivered with the router. But I hadn't though of that. There's a million other transformers and electrical devices connected in the house but of course they don't feed the router. But they still could generate RF interference I assume. Perhaps I need to start unplugging stuff...

Evening time, back in my usual location. Streaming YouTube to the usual tablet, to an old MacBook and to the XPS13 itself. Pinging from XPS13 to router, connected on 2.437GHz (802.11n). Using the microwave oven as well. Latencies look good, the occasional 100-300ms latency (often for more than one ping so again some temporal correlation) but mostly 1-3ms. Turned on plasma TV and set top box. Still OK.

I didn't do any configuration changes to the router/AP but auto selection could have used a different channel (iwconfig shows same frequency as before though)? Normally, I can only see one or two neighbouring WLAN's and they are a lot weaker than my own.

Is there an external source of this potential RF interference? A neighbour using some electric equipment? (often in the evenings but only for a few seconds at a time).

Let's hope it stays good.

There are many potential sources of interference. Without significant investment in hardware equipment to be able to 'see' this in your environment, it remains invisible but still affects our wireless networks.

If interference was causing malformed wireless packets, would this not increase the error rate shown in either /proc/net/wireless or the router's "wl pktcnt" command?

I just ran the following commands on my router via SSH. Perhaps compare them to your results?


and here are some wireless network stats from the Nexus 7 (2012 with Android 5.1) that I am using to type this reply;

Don't unplug stuff.

Did you ever change channels on the router and disable auto-select?
Try channel 1 and 11 and see how they differ.

Sorry for such a late response.

I finally got around to changing the channel. For some reason, I returned to using the TL-WR1043ND (2.4GHz only) with OpenWRT. By letting the router select the channel automatically, it chose channel 11 which was the most congested channel (at least three other AP's in my neighborhood). I did a channel/AP scan and then selected channel 4, the closest channels already in use were 1 and 8. Ping latencies now look very good, avg < 5ms, the occasional latency up to 100ms. None of these 5xxx, 4xxx, 3xxx, 2xxx, 1xxx, xxx ms ping latencies.

So the conclusion is that 2.4GHz channel 11 sometimes has a lot of interference which prevents stations from transmitting for many seconds. Somehow ping packets are enqueued somewhere and are eventually released, probably due to some timeout and retransmission (on some level, probably not in the ping application or OS).

likely a non-wifi signal. baby monitor, etc.

You might try running a respectable WiFi scanner, like kismet, that can catalogue the interference, on all channels, over a day, week or however you run it for. You could probably download some Live-CD like Kali Linux and run it on some old computer for a day or ten. It will watch for all sorts of problems, like little hacker-wannabes scanning your neighborhood or another interefering WiFi station that's hidden.

It could be interesting to at least see what is happening around you...

A Simple WiFi scanner may not, or never, reports non-WiFi signals like bahy monitor analog cameras.

Not scanning will assure that you get zero data.

I remember hearing that a very cheap software-defined radio had came out. I wonder if the technology has matured enough to be useful in this field.

scanning for non-WiFi and WiFi signals takes
1) a good spectrum analyzer with a low noise amplifier
2) proper antenna
3) ideal location
4) Hours and hours of monitoring and automated logging

Even so, in these unlicensed bands, what you see now will be different next month!

stevech said:

scanning for non-WiFi and WiFi signals takes
1) a good spectrum analyzer with a low noise amplifier
2) proper antenna
3) ideal location
4) Hours and hours of monitoring and automated logging

Even so, in these unlicensed bands, what you see now will be different next month!

Click to expand...

Hey, as long as it is as entertaining as watching Windows defragment, I'm in!

This likely is not a Wireless issue, but a basic network issue... and something that is causing the routers to get so loaded up CPU wise... those ping times are insane for a Local Area Network, period..

My thoughts...

1) Swapped Routers, problem didn't change
2) Changed Clients, problem didn't change
3) Changed Channels, a little bit of change

What hasn't changed?

I'm willing to bet that the same cable is being used from the Modem to the Router/AP, and that's the first thing I would change out there - also, check the upstream modem - it might be in a funky state...

Swap out the WAN cable, and reset the modem... report back.

sfx