Proposal for a new test

[Zepplock]

Hello All,

I've been reading articles here for some time (and forums too) and finally decided to subscribe. You guys are doing great job here!!!

Preface:
I'm a professional software developer (mostly high performance servers and networking). Recently I had a project where I had to develop a system to manage wireless sensors (large amount of them). Part of the project was to choose and deploy wireless access points. People I was working with claimed that they can buy $30-$100 ones at Frys/Newegg and run with them. I was very skeptical.

Test:
They bough like 10 different ones and tested them like this: a dozen of G wireless clients connecting to the wifi access point, and accessing an HTTP server every minute or two (simple HTTP GET).

Results:
Not a single access point could reliably pass the test. Issues encountered:
- Unreliable dhcp servers (can't get IP address)
- weird delays in getting a wifi link (200ms usually but can take up to 2-3 seconds, probably due to WEP/Security?)
- hanging up, requiring a hard reset
- signal dropouts (can't get a wifi link)

Proposal:
Devise a test which can stress test access points for at least a week by simulating a dozed of wifi devices with different patterns:
- frequent on/off, requiring a wifi link negotiation several times an hour
- ocasional file transfer (max bandwidth)
- movie watching (mid-level bandwidth) but for 2 hours
- web browsing (bursts of traffic with several tcp connections open)
etc

 

[Terry Kennedy]

- Unreliable dhcp servers (can't get IP address)

If these were really access points and not routers, wouldn't that indicate a problem with the router and not the access point? Unless it was actually an inability to get a packet to the DHCP server and back, due to an access point problem.

Devise a test which can stress test access points for at least a week by simulating a dozed of wifi devices with different patterns:

An interesting idea. I can think of a couple issues, though. First, it would be difficult to coordinate multiple clients to test APs in a repeatable manner. Next, a particular pattern might cause one AP to have difficulties but doesn't affect a different AP (which might have a more severe problem, just not detected by this test pattern). Then all tests would need to be repeated with the enhanced set of patterns, taking a week per device - probably not practical.

Another issue is that revew samples are often early production units and the firmware (and sometimes even the hardware) may be changed by the time the unit is available at retailers. So the test would have to identify the specific firmware in use, which might be a test version not available to purchasers. Hopefully, bugs will be fixed as time goes on, though as we've seen sometimes new problems are introduced. Then there's the situation where the product model number doesn't change, but what is in the box (both hardware and software) is very different. Look at the Linksys WAP54/WRT54 products for a very big example.

Probably the best bet for a home user who doesn't need to have the latest and greatest is to ask what products other users have had success with (and find out the software version, too!) and try that.

For a business environment where you need to buy X identical units TODAY, you may be better-served by getting commerical-grade units from one of the major players. Of course, that means spending a lot more money (sometimes a LOT more). But you do get useful phone support if things go wrong. Which can certainly happen - even the major players like Cisco aren't immune. But the support tends to be better.