2.5G, LAN aggregation, switch - oh my!

[TyantA]

• My Asus AX86U has a 2.5G port and supports LAN aggregation
• My gigabit switch supports LACP
• I picked up a 2.5G network card to toy with

My Unraid box and my desktop currently have 1G NICs. House is wired with CAT 6.

I'm trying to wrap my head around the topology to maximize read/write between my main desktop and unraid server.

Originally I was thinking about installing the 2.5G card in the server, bypassing the switch, plugging straight into the router.
Now in order to use that additional bandwidth, I was thinking of enabling a aggregated link (router, ports 1 & 2) to my switch.

1. Is that it? I guess if I stop there, in theory, multiple devices accessing the unraid box via my switch would benefit from the added bandwidth, but not any particular system.
2. I guess I'd need to 'continue' the aggregation straight to my desktop by adding another 1G card to it if I wanted to see >1G transfers?

If #2 is the case, I may need to rethink as I just got finished hiding my one CAT 7 connection to my desktop through closets, under carpets, etc. I'm not running another. I suppose I could do the reverse: put the 2.5G card in my desktop for the Cat 7 connection I have and aggregate two 1G connections on the server side... but it's more likely the added bandwidth (of the 2.5G card) would be more useful on the server side for the rest of the wired clients in the house.

Thoughts?

 

[Tech9]

If your ISP is <Gigabit - keep the router you already have and connect your 2.5GbE capable devices to 2.5GbE switch for faster LAN.

If your ISP is >Gigabit - the configuration above plus preplace your router with 2x 2.5GbE ports model, one for WAN and one for LAN.

Link Aggregation doesn't work well in Asuswrt. Don't waste too much time with it.

Home routers don't really have 2.5GbE capable hardware. You have to be careful what firmware options are enabled.

 

[Tech Junky]

I was running LACP on my CM to get beyond 1gbps and bundled 2 x 1ge ports to it and worked just fine. Max speed hit ~1500mbps.

If you want internal access for the LAN to hit those higher speeds your cabling is fine. You'll want to upgrade the NIC or use a dongle on each device to hit 2.5gbps speeds. NICs are cheap it's the switch that will cost you a bit more but it's only $100-$150 for a switch that's unmanaged.

ISP <2.5> Router <2.5> Switch <> PC's

 

[TyantA]

If your ISP is <Gigabit - keep the router you already have and connect your 2.5GbE capable devices to 2.5GbE switch for faster LAN.

If your ISP is >Gigabit - the configuration above plus preplace your router with 2x 2.5GbE ports model, one for WAN and one for LAN.

Link Aggregation doesn't work well in Asuswrt. Don't waste too much time with it.

Home routers don't really have 2.5GbE capable hardware. You have to be careful what firmware options are enabled.

Definitely < Gigabit. I live in the boons

Thanks for the caution about not wasting too much time. I'm getting the sense that Asus tries to "tick all the boxes" from a marketing perspective but execution is lacking - is an ongoing trend with them. (I'm also trying to use load balancing for dual WAN with marginal success).

I didn't qualify my question with the goal of trying to 'maximize with what I already have'. It's not (currently) a big enough need that I would invest in other hardware (other than considering picking up a dedicated load balancer given the aforementioned dual WAN needs and sub-par Asus execution). I.e. it'll be a while before I pick up a 2.5G switch... but other than that I would agree with your recommendation, thank you.

 

[Tech9]

I'm getting the sense that Asus tries to "tick all the boxes" from a marketing perspective but execution is lacking

Correct. Asus is mostly working on what sells more routers - AiMesh. Some of the other advertised features are broken for years.

I'm also trying to use load balancing for dual WAN with marginal success

Dual WAN is also not working properly. Load balancing is very basic and breaks connections. Fail over/back may or may not happen.

 

[TyantA]

I was running LACP on my CM to get beyond 1gbps and bundled 2 x 1ge ports to it and worked just fine. Max speed hit ~1500mbps.

If you want internal access for the LAN to hit those higher speeds your cabling is fine. You'll want to upgrade the NIC or use a dongle on each device to hit 2.5gbps speeds. NICs are cheap it's the switch that will cost you a bit more but it's only $100-$150 for a switch that's unmanaged.

ISP <2.5> Router <2.5> Switch <> PC's

Thanks for sharing your experience.

It is internal LAN speeds / overall bandwidth I'm looking to improve if I can.

Basically, I have 3 servers in the works. Main, work and backup.

I know the Asus manual says you can only aggregate 2x 1G ports: 1 & 2; you cannot aggregate more than that. But what occurred to me is that I should be able to aggregate links to the other servers from the switch as well.

I 100% see that picking up a 2.5G switch is the obvious route, but given that I have a managed switch (that I've never used - so that's new to me), plenty of ports and tons of CAT6 cable... figured why not give it a try?

As I've thought about this, I think my use case has shifted a bit. I won't worry about faster-than-1G on my desktop for now. However, if I can put:

1. Primary server on the 2.5G port.
2. Ports 1 & 2 aggregated > switch
3. switch > work server (2x 1G, aggregated)
4. switch > backup server (2x 1G, aggregated)

Then in theory, say a backup running from primary to backup for example wouldn't be as impacted by other network requests? Am I thinking that through correctly?

I have plenty of gigabit NICs lying around so I really wouldn't have to buy anything.

 

[Tech9]

What the servers are going to serve that much? My NAS is potentially the highest bandwidth device and it can do 70MB/sec over Wi-Fi 5 with Gigabit LAN port. What more is needed for home use? If you are going to do business from home - invest in some better equipment and call it a day.

 

[Tech Junky]

1. Primary server on the 2.5G port.
2. Ports 1 & 2 aggregated > switch
3. switch > work server (2x 1G, aggregated)
4. switch > backup server (2x 1G, aggregated)

It's totally up to you.

I would personally put in a 5GE quad port in the primary server and then a single 5GE in the 2 others.

Your average 3.5" drive these days does 220MB/s and if you're doing Raid 0/10 that boosts you to 400MB/s+

Tie the 3 together and that leaves 2 ports to uplink to the router. If you decide to roll your own router off the quad port server you have the bandwidth to do up to a 5GE ISP if they come out to the sticks and if they don't your backups will process much quicker w/ less chance of corruption due to the length of time it takes them to process.

If you want to use what you have on hand and not make the investment then it depends on how the switch handles bundling of interfaces. There's 4 different modes of aggregation available and they all handle data differently.

I run my setup in R10 and get ~440MB/s out of the drives and the 5GE link has room for another 200MB/s if needed when a backup or copy is being done at full speed. My bo also serves as my router / AP / firewall and some other functions as well. I have 3 wired devices and everything is WIFI otherwise. I have 1 5GE open on the card I'm using and a 2.5GE port on the MOBO that's also available as a backup.

 

[TyantA]

What the servers are going to serve that much? My NAS is potentially the highest bandwidth device and it can do 70MB/sec over Wi-Fi 5 with Gigabit LAN port. What more is needed for home use? If you are going to do business from home - invest in some better equipment and call it a day.

Home + business; on some days there will be more than one person working from here as well.

Truth be told, there's nothing "wrong" with gigabit connections everywhere, it just so happens I'm finally installing my patch panel and plan to set up this 24 port managed switch so I thought I'd look at what else I could eek out of it at the same time. So less of a need, more of a "it's what I have, let's see what I can do with it.

The work server will be new (to the rack) and the backup server is something I've been wanting to execute on for 5+ years and just need to make it happen. So these are new "needs" that I have yet to experience how day-to-day will be impacted.

 

[TyantA]

It's totally up to you.

I would personally put in a 5GE quad port in the primary server and then a single 5GE in the 2 others.

Your average 3.5" drive these days does 220MB/s and if you're doing Raid 0/10 that boosts you to 400MB/s+

Tie the 3 together and that leaves 2 ports to uplink to the router. If you decide to roll your own router off the quad port server you have the bandwidth to do up to a 5GE ISP if they come out to the sticks and if they don't your backups will process much quicker w/ less chance of corruption due to the length of time it takes them to process.

If you want to use what you have on hand and not make the investment then it depends on how the switch handles bundling of interfaces. There's 4 different modes of aggregation available and they all handle data differently.

I run my setup in R10 and get ~440MB/s out of the drives and the 5GE link has room for another 200MB/s if needed when a backup or copy is being done at full speed. My bo also serves as my router / AP / firewall and some other functions as well. I have 3 wired devices and everything is WIFI otherwise. I have 1 5GE open on the card I'm using and a 2.5GE port on the MOBO that's also available as a backup.

Cache pools are using SSDs for better throughput.

"it depends on how the switch handles bundling of interfaces. There's 4 different modes of aggregation available and they all handle data differently." Thank you, THIS is what I now realize I need to focus on/research next.

 

[Tech Junky]

Cache pools are using SSDs for better throughput.

Doesn't fix the bottleneck the port speed invokes. Network end to end needs to meet the speed of the disks if you want full speeds. Bundling 2 interfaces is a good leap in speed but still less than or about the same speed as a single spinner. SSD speeds would need 4 links per drive to max them out.

 

[Tech9]

So less of a need, more of a "it's what I have, let's see what I can do with it.

After you collect 100kg junk in your rack just because you got it cheap you’ll realize you don’t need it for home use. I downsized a home lab to equipment I can carry in one bag and the user (family) experience is exactly the same. No more power hungry old equipment and fan noise. I currently have Netgate firewall, 16-port Netgear PoE switch, 4x Ruckus PoE access points and a small mini-PC based NAS with 2x 8TB hard drives. All the junk out, under 100W total.

 

[ddaenen1]

After you collect 100kg junk in your rack just because you got it cheap you’ll realize you don’t need it for home use. I downsized a home lab to equipment I can carry in one bag and the user (family) experience is exactly the same. No more power hungry old equipment and fan noise. I currently have Netgate firewall, 16-port Netgear PoE switch, 4x Ruckus PoE access points and a small mini-PC based NAS with 2x 8TB hard drives. All the junk out, under 100W total.

Exactly the same process on my end. I still think i am running heavy but already a lot better than the MD1000 and 3 different servers with each one application running. I am thinking to get rid of the Synology too and integrate this into the TrueNAS server that currently only runs Plex and Nextcloud and replace the LFF SAS Drives by SSD's.

 

[follower]

• My Asus AX86U has a 2.5G port and supports LAN aggregation
• My gigabit switch supports LACP
• I picked up a 2.5G network card to toy with

My Unraid box and my desktop currently have 1G NICs. House is wired with CAT 6.

I'm trying to wrap my head around the topology to maximize read/write between my main desktop and unraid server.

Originally I was thinking about installing the 2.5G card in the server, bypassing the switch, plugging straight into the router.
Now in order to use that additional bandwidth, I was thinking of enabling a aggregated link (router, ports 1 & 2) to my switch.

1. Is that it? I guess if I stop there, in theory, multiple devices accessing the unraid box via my switch would benefit from the added bandwidth, but not any particular system.
2. I guess I'd need to 'continue' the aggregation straight to my desktop by adding another 1G card to it if I wanted to see >1G transfers?

If #2 is the case, I may need to rethink as I just got finished hiding my one CAT 7 connection to my desktop through closets, under carpets, etc. I'm not running another. I suppose I could do the reverse: put the 2.5G card in my desktop for the Cat 7 connection I have and aggregate two 1G connections on the server side... but it's more likely the added bandwidth (of the 2.5G card) would be more useful on the server side for the rest of the wired clients in the house.

Thoughts?

I hate normal CAT.6 cable.