1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

New Router Test Processes Are Coming

Discussion in 'LAN & WAN Article Discussions' started by thiggins, Apr 25, 2016.

  1. sfx2000

    sfx2000 Part of the Furniture

    Joined:
    Aug 11, 2011
    Messages:
    14,340
    Location:
    San Diego, CA
    I'm sure you've reached out to the suppliers - both for the test equipment and Vendors/OEM's on feedback - but as you state, it's the WAN connectivity that is problematic... but if one were to stand up a small DCHP6-PD "WAN" server, this might help - at least to get a baseline performance, as this is fairly common (it's the options that are tough across different operators, Comcast and Cox have both had issues here - Cox's is ongoing even now, as they pretty much rolled out v6 in the last couple of months)
     
  2. thiggins

    thiggins Mr. Easy Staff Member

    Joined:
    May 18, 2008
    Messages:
    14,371
    CDRouter can test all the IPv6 connection types. It's the equipment that is inconsistent. Not all support PD, for example.

    Give it another couple of years....
     
    sfx2000 likes this.
  3. pete y testing

    pete y testing Very Senior Member

    Joined:
    Jul 12, 2011
    Messages:
    1,931
    Location:
    victoria, australia
    any chance of testing in the windows environment with standard samba copy and paste as well as icharriot results , i know its a lot of extra work but its also a better indication of how the router works in a standard user environment and what the end use can expect to see when the just copy and paste between devices

    pete
     
  4. sfx2000

    sfx2000 Part of the Furniture

    Joined:
    Aug 11, 2011
    Messages:
    14,340
    Location:
    San Diego, CA
    I think part of the challenge these days - what is a standard SMB environment?

    Windows Clients/Servers is a mixed bag - OSX Client to Linux SMB3 server on a NAS? No Windows there, mate...

    iPerf UDP/TCP numbers are probably a better benchmark of actual performance...
     
  5. pete y testing

    pete y testing Very Senior Member

    Joined:
    Jul 12, 2011
    Messages:
    1,931
    Location:
    victoria, australia
    but to the average man on the street jipref doesnt make a lot of sense apart from one being higher than another , and dont really give the average user an idea of what they can expect when they transfer from one device to another

    i know the jipref is the preferred method on snb but a simple benchmark set in MB/s would be beneficial to the readers and to the overall review

    just my point of view

    pete
     
  6. sfx2000

    sfx2000 Part of the Furniture

    Joined:
    Aug 11, 2011
    Messages:
    14,340
    Location:
    San Diego, CA
    all numbers are relative though... just my thoughts...
     
  7. stevech

    stevech Part of the Furniture

    Joined:
    Mar 13, 2010
    Messages:
    7,007
    Location:
    San Diego
    Started hearing that from DoD and others some 15 years ago.
    Will the barrage of IoT thingies all be NATed too? I think so.
     
  8. pete y testing

    pete y testing Very Senior Member

    Joined:
    Jul 12, 2011
    Messages:
    1,931
    Location:
    victoria, australia
    true but how they relate to the end user is the key
     
  9. System Error Message

    System Error Message Part of the Furniture

    Joined:
    Oct 14, 2014
    Messages:
    4,204
    The only problem with star tech is that they stuff is usually overpriced for the quality it provides. If you are worried about emf than you need a metal/aluminium case for the drive. The only quality star tech product i have is an open and adjustable server rack on wheels and even that was very expensive.

    Esata is still used around. On some desktops you have an esata port that connects to motherboard SATA. It basically connects an internal sata to external so you dont get usb overheads depending on if sata is connected to chipset or some extra chip. Its too bad a lot of manufacturers didnt adopt esata but part of the problem was that you needed another power source compared to usb and by the time powered esata came out you already had usb3 and all people want are convenience and dont seem to care about overhead and that extra performance.

    Aside from performance testing feature testing should be done too such as with ipv6 and mixed ipv6 and ipv4 environments. Bug testing where if every feature was used, would the router hang after being fully loaded for a while? Dont forget to test reliability.
     
  10. thiggins

    thiggins Mr. Easy Staff Member

    Joined:
    May 18, 2008
    Messages:
    14,371
    If you are referring to the storage throughput tests, Windows Robocopy is used
    http://www.smallnetbuilder.com/lanwan/lanwan-howto/32612-how-we-test-router-storage-performance

    iperf/Jperf has not been used in any of the current benchmarks.
     
  11. pete y testing

    pete y testing Very Senior Member

    Joined:
    Jul 12, 2011
    Messages:
    1,931
    Location:
    victoria, australia
    well i was more wanting wireless throughput also measured that way as well
     
  12. thiggins

    thiggins Mr. Easy Staff Member

    Joined:
    May 18, 2008
    Messages:
    14,371
    Sorry, that is not going to happen.
     
  13. GJJ

    GJJ New Around Here

    Joined:
    Apr 26, 2016
    Messages:
    3
    After reading a post about different h/w revisions for a particular router and what the different chipsets each revision brought to the table, I'd like to see testing of concurrent activities. As it was said in the initial article, we're using our routers for a lot of different things now days, but the way I have read the tests, is that they're only testing 1 aspect of the router at a time.

    The post was about different revisions for the Asus RT-AC68U (excuse me if I remember the wrong: it's mostly for example purposes.)
    A1/2 - Base config
    B1 - Updated cpu running 200MHz faster
    C1 - Not released yet, but supposedly with different wifi chips that offload some of the work from the main cpu

    In the tests that have been used until now, I wouldn't expect to see much if any significant difference between rev. B1 and C1.
    However if there was a load test, that was loading up the wifi components and the routing components at the same time, I'd expect to see a more noticeable improvement with the newer C revision.

    Some sort of mixed load test would also show up differences between single and multi core routers when some users are doing LAN-WAN work, while others are accessing a storage device connected to the router via USB.

    Agreed, there are a lot of different router hardware combinations out there, but some "simple" mixed load testing could show up differences in hardware and firmware configurations. (I say "simple", but I doubt the tests would be simple to standardise and run.)

    Anyway, just an idea.

    Greg
     
  14. thiggins

    thiggins Mr. Easy Staff Member

    Joined:
    May 18, 2008
    Messages:
    14,371
    Wireless load testing using the Veriwave system is planned. It may not be in the initial process release.
     
  15. GJJ

    GJJ New Around Here

    Joined:
    Apr 26, 2016
    Messages:
    3
    Cool thanks.

    I saw that in your initial article, but wasn't sure what was meant by it.

    I'll be interested to see it when it arrives, even if not initially.

    Greg
     
  16. bman212121

    bman212121 New Around Here

    Joined:
    Oct 21, 2015
    Messages:
    6
    Which is likely part of the problem. I've been consistently just disabling IPv6 on everything I've setup because it just doesn't work. When all of your networking / sys admin contacts are doing the same and going out of their way to disable a feature, there certainly is a problem.
     
  17. mactenchi

    mactenchi Occasional Visitor

    Joined:
    Apr 27, 2016
    Messages:
    14
    Tim, any chance for a long-term component to testing? I find performance/features degrade over time. For example, my Archer C7 seems to need a hard restart periodically to fix NAT.
     
  18. thiggins

    thiggins Mr. Easy Staff Member

    Joined:
    May 18, 2008
    Messages:
    14,371
    Unlikely. Most I could do would be to loop the functional tests overnight. Doubt that would help much.
     
  19. L&LD

    L&LD Part of the Furniture

    Joined:
    Dec 9, 2013
    Messages:
    12,594
    You could ask the manufacturers for a few dozen units to send to your closest friends for 'long term testing', can't you? :D

    (No, I'm not volunteering or suggesting we're actual 'friends'). ;)
     
  20. mactenchi

    mactenchi Occasional Visitor

    Joined:
    Apr 27, 2016
    Messages:
    14
    Might be interesting to try, see if results change. Probably not worth adding to the test process, though, you're right.