Trying to plan best use of router & switches on home network

[awediohead]

I've managed to install OPNsense on a Fujitsu S920 and I'm wondering how to make best use of the 5 ports I have available. I also have a 16 port Netgear plus switch (GS116E) which is a managed switch.

Neither of these are actually in use currently - I've got a lot of wiring to do from four rooms, through the attic (it's a bungalow) to a patch panel in a central closet where the rest of the gear and my server will eventually be installed.

The router has an onboard single port Realtek 1Gb NIC and an Intel based four port 1Gb PCIe NIC.

For now I've set up the router to use the onboard Realtek NIC for WAN (since our internet is less than 100Mbps) with the idea that I'd then have the Intel x4 Gb ports to play with for LAN connections.

One of the (many) things I'm not clear on is whether I'd be better off keeping both WAN and LAN on the same Intel based NIC and ignore the onboard NIC?

I've just no idea how to weigh up the pros and cons or even whether it makes any difference. Just lots of internet opinions about Intel being generally more dependable than Realtek - though my experience over years of having Realtek NICs on various motherboards is I've not noticed any difference.

I was thinking that if I use the Realtek NIC for WAN then I could have one port of the x4 for LAN to the switch, two ports connected as a LAGG to the switch and maybe the last port connecting directly to an AP.

Again no idea id there's any benefit in having the AP connected directly to the router vs connecting it through the switch?

Eventually I'll have two ethernet runs from each of four rooms (making eight in all) wired to a patch panel and from that to the 16 port Netgear switch.

While I can and doubtless will tweak things after the wiring is done and we're actually using this hardware - I'd prefer to have a reasonably solid plan for how all this will fit together in advance.

My wife is housebound by her disability and so very reliant on a solid internet connection.

She'll manage perfectly well for a couple of hours without internet while I install and connect stuff up, but ideally I'll have it all configured and ready to swap out for our existing (off the shelf all in one router/wifi) set up by running everything in parallel and testing what I can before taking the plunge.

Any advice gratefully received - I'm very likely not to have even thought about a bunch of things I should be considering at this stage as I'm very new to all this!
Cheers

 

[Tech Junky]

RTK v Intel = wash until you want to do certain things

I would definitely LAG things for redundancy but also for speed back to your rack if you need to bundle switch ports as well for a specific device.

AP off the router... doable if you need the AP in that location. If not then put it where it needs to be. For the AP you can plug it into the existing network and turn down the AIO WIFI to test.

To minimize down time doing the AP will keep things up and running w/ little interruption as you can have both running at the same time until you turn it off on the router side.
Preconfigure the new box as much as possible which in this case it sounds like mostly setting the WAN/LAN groups and/or LAG to the switch.
Preconfigure the switch with the LAG ports and test with the new box prior to cutting over LAN devices. The existing router can be plugged in at this point as well if you shut off the DHCP on the new box or set it to a different subnet to take over when the AIO expires.

All in all though with planning the cutover could be a little as 3-5 minutes while the new router boots or less if all you have to do is move the Ethernet cable from one box to the other and reboot the ISP device to get a new IP.

 

[drinkingbird]

I've managed to install OPNsense on a Fujitsu S920 and I'm wondering how to make best use of the 5 ports I have available. I also have a 16 port Netgear plus switch (GS116E) which is a managed switch.

Neither of these are actually in use currently - I've got a lot of wiring to do from four rooms, through the attic (it's a bungalow) to a patch panel in a central closet where the rest of the gear and my server will eventually be installed.

The router has an onboard single port Realtek 1Gb NIC and an Intel based four port 1Gb PCIe NIC.

For now I've set up the router to use the onboard Realtek NIC for WAN (since our internet is less than 100Mbps) with the idea that I'd then have the Intel x4 Gb ports to play with for LAN connections.

One of the (many) things I'm not clear on is whether I'd be better off keeping both WAN and LAN on the same Intel based NIC and ignore the onboard NIC?

I've just no idea how to weigh up the pros and cons or even whether it makes any difference. Just lots of internet opinions about Intel being generally more dependable than Realtek - though my experience over years of having Realtek NICs on various motherboards is I've not noticed any difference.

I was thinking that if I use the Realtek NIC for WAN then I could have one port of the x4 for LAN to the switch, two ports connected as a LAGG to the switch and maybe the last port connecting directly to an AP.

Again no idea id there's any benefit in having the AP connected directly to the router vs connecting it through the switch?

Eventually I'll have two ethernet runs from each of four rooms (making eight in all) wired to a patch panel and from that to the 16 port Netgear switch.

While I can and doubtless will tweak things after the wiring is done and we're actually using this hardware - I'd prefer to have a reasonably solid plan for how all this will fit together in advance.

My wife is housebound by her disability and so very reliant on a solid internet connection.

She'll manage perfectly well for a couple of hours without internet while I install and connect stuff up, but ideally I'll have it all configured and ready to swap out for our existing (off the shelf all in one router/wifi) set up by running everything in parallel and testing what I can before taking the plunge.

Any advice gratefully received - I'm very likely not to have even thought about a bunch of things I should be considering at this stage as I'm very new to all this!
Cheers

At 100M I don't think there's really any issue with the Realtek for WAN. The intel add-in card likely has much better processing power but probably not an issue at that speed. But if you don't need all 4 ports, it may make sense to just leave the onboard unused.

You need to decide how you want to segment your network, then from there decide how to connect things up. If you want your smart switch to handle all the VLAN distribution then just do a 4 port LAG from opnsense to switch and plug everything else into the switch, trunk the VLANs across the LAG and distribute out to your wired devices and access points from there.

If you connect an AP directly to the opnsense it would kind of only make sense to do that if it is to be an AP on a dedicated isolated subnet. The bridging performance of that 4 port NIC probably won't be as good as the switch and honestly it is sort of a waste of a port and processing power on the pfsense.

But like I said, what (if any) segmentation/isolation are you looking for? If none, and you only have 100M internet, then that opnsense box is kinda overbuilt for what you need. Probably start with a logical block diagram of what isolated (or filtered/firewalled) segments you want then decide the best way to accomplish that.

 

[coxhaus]

I don't know anything about a Fujitsu S920. If you have Intel ports use them over the realtek.
Only use a Lagg if your speed is going to exceed the speed of the port. Laggs can chew up a lot of ports that can be used for something else. If a Lagg forces, you into a second switch you may be better off using 1 switch with 1 backplane and no Lagg. It depends. You need to analyze traffic flows on what best fits you.

 

[awediohead]

All in all though with planning the cutover could be a little as 3-5 minutes while the new router boots or less if all you have to do is move the Ethernet cable from one box to the other and reboot the ISP device to get a new IP.

Thanks for the advice - That's exactly what I was hoping, at least with regard to the wired side of things.
Because all this is being done on a tight budget - initially I'll use our current router (an Asus RT-AC86U) in AP only mode. I just want to plan for it to be eventually replaced with a dedicated AP at 6e speeds

 

[Tech Junky]

AP at 6e speeds

6/6E speeds are the same. Max of 1.2gbps unless you upgrade your client wifi cards to AX411's in which they combine 2.4/5 bands into a single pipe up to 1.5gbps. Waiting at this point for BE class would be a better idea. It doubles the 6ghz spectrum to 320mhz which could jump LAN speeds to ~3gbps or higher.

If your current "AP" is the AC class then going from AC >> AX will be an improvement of 50% but, you also have to upgrade the clients to AX210/AX211 cards to realize the speed increase. The cards aren't all that expensive at ~$30/ea but, it's an added cost. The AX411 though can be had for under $20/ea if you have ADL/RPL CPUs in the devices to take advantage of the CNVIO system they rely on.

For PC's it's hard to find much that has a BE card and there are no cards to be bought at the moment. However on the Phone / Tablet side there are a couple of devices that have it already. There's 2-3 routers out now that have BE as well but, they're way over priced because they're "mesh" systems anywhere from $500-$1500.

If you want to save on 6/6E then look at Zyxel as for $150 you can get either option. The NWA210AX has 2.4/5 and the NWA220AX-E has 2.4 + 5 or 6. Gong up to a true tri-band AP though bumps the price to about $275. It depend son the mix of devices and whether or not the priority devices have 6ghz as an option and kicking everything else down to 2.4/300mbps cap.

 

[awediohead]

You need to decide how you want to segment your network, then from there decide how to connect things up. If you want your smart switch to handle all the VLAN distribution then just do a 4 port LAG from opnsense to switch and plug everything else into the switch, trunk the VLANs across the LAG and distribute out to your wired devices and access points from there.

I'm hoping to set up audio over IP recording/monitoring between each of the four rooms - this is why (apart from redundancy) there will be two ethernet drops to each of our four rooms.

One will be for standard PC to LAN to WAN internet access and probably go to a little 5 port unmanaged switch in each room. The second cable to each room being on a separate VLAN dedicated to the audio recording side of things, where minimising latency will be highly desirable so as direct a connection between PC's as possible.

I'm also hoping that we may get some support to "smarten up" what is currently an exceptionally dumb home with some smart adaptations that would improve my wife's quality of life as a disabled person. So for what we use right now, yes the whole thing is overkill - despite having been done on a very tight budget - however hopefully it will be a solid foundation for future upgrades, certainly with enough flexibility to add in a VLAN to isolate devices (we don't currently have any) that might try to phone home unless they're locked down.

 

[drinkingbird]

I'm hoping to set up audio over IP recording/monitoring between each of the four rooms - this is why (apart from redundancy) there will be two ethernet drops to each of our four rooms.

One will be for standard PC to LAN to WAN internet access and probably go to a little 5 port unmanaged switch in each room. The second cable to each room being on a separate VLAN dedicated to the audio recording side of things, where minimising latency will be highly desirable so as direct a connection between PC's as possible.

I'm also hoping that we may get some support to "smarten up" what is currently an exceptionally dumb home with some smart adaptations that would improve my wife's quality of life as a disabled person. So for what we use right now, yes the whole thing is overkill - despite having been done on a very tight budget - however hopefully it will be a solid foundation for future upgrades, certainly with enough flexibility to add in a VLAN to isolate devices (we don't currently have any) that might try to phone home unless they're locked down.

Technically even cheap switches nowadays have very little latency associated so you could run one wire with VLANs and a smart switch in each room, but if you're running them anyway, running two isn't a bad idea.

In reality, if you want the lowest latency and that audio network does not need access to anything else, plugging those drops directly into an isolated dumb switch is going to be your lowest latency, no adding/stripping of VLAN tags etc. If that network doesn't need access to anything else, just leave it like that. Or if it does (or you want it to have DHCP etc) then plug a cable from that dumb switch into a dedicated port on the opnsense with no vlans.

On the 86U you can make use of the built in VLAN 501 and 502 that get created when you enable guest wireless 1 with LAN access disabled. Those two VLANs automatically gets tagged out of all LAN ports (and I believe the WAN port when in AP mode) along with the main VLAN 1 untagged. Trunk that on your switches to the opnsense box and you've now got your isolation which you can also put wired devices in as well. Opnsense now controls all your traffic between the networks and you can create rules to allow exactly what you do and don't want.

When it comes time to upgrade your AP I'd look for one that is truly VLAN aware (user configurable) giving you a bit more control but for now that will work fine.

A 2 or 4 port LAG to the opnsense is not unreasonable but only if you think you'll need more than a gig going between the networks/VLANs. LAG can add a bit of latency so I wouldn't run the audio network across it, but for everything else it shouldn't be noticeable (probably wouldn't be for audio but as you say, no sense in introducing any extra latency, or more importantly, jitter). If you don't need more than a gig right now, probably no sense in adding complexity for nothing.

You could plug the AP directly into the opnsense but then you'd need to bridge that port/vlans to your switch via the intel nic, not really the ideal design in my mind. Unless it needs no access to the LAN and only internet then it might make sense, but I'm assuming that isn't the case.

 

[awediohead]

Technically even cheap switches nowadays have very little latency associated so you could run one wire with VLANs and a smart switch in each room, but if you're running them anyway, running two isn't a bad idea.

In reality, if you want the lowest latency and that audio network does not need access to anything else, plugging those drops directly into an isolated dumb switch is going to be your lowest latency, no adding/stripping of VLAN tags etc. If that network doesn't need access to anything else, just leave it like that. Or if it does (or you want it to have DHCP etc) then plug a cable from that dumb switch into a dedicated port on the opnsense with no vlans.

On the 86U you can make use of the built in VLAN 501 and 502 that get created when you enable guest wireless 1 with LAN access disabled. Those two VLANs automatically gets tagged out of all LAN ports (and I believe the WAN port when in AP mode) along with the main VLAN 1 untagged. Trunk that on your switches to the opnsense box and you've now got your isolation which you can also put wired devices in as well. Opnsense now controls all your traffic between the networks and you can create rules to allow exactly what you do and don't want.

When it comes time to upgrade your AP I'd look for one that is truly VLAN aware (user configurable) giving you a bit more control but for now that will work fine.

A 2 or 4 port LAG to the opnsense is not unreasonable but only if you think you'll need more than a gig going between the networks/VLANs. LAG can add a bit of latency so I wouldn't run the audio network across it, but for everything else it shouldn't be noticeable (probably wouldn't be for audio but as you say, no sense in introducing any extra latency, or more importantly, jitter). If you don't need more than a gig right now, probably no sense in adding complexity for nothing.

You could plug the AP directly into the opnsense but then you'd need to bridge that port/vlans to your switch via the intel nic, not really the ideal design in my mind. Unless it needs no access to the LAN and only internet then it might make sense, but I'm assuming that isn't the case.

That gives me a much clearer idea as to what to prioritise and what to factor in when planning things. Thank you hugely for taking the time to explain it all - it's so helpful when someone has the depth of knowledge to not only answer questions, but provide answers to the questions I didn't know I needed to ask!

Thank you again!

 

[drinkingbird]

That gives me a much clearer idea as to what to prioritise and what to factor in when planning things. Thank you hugely for taking the time to explain it all - it's so helpful when someone has the depth of knowledge to not only answer questions, but provide answers to the questions I didn't know I needed to ask!

Thank you again!

Only caveat is I know the Asus will have the VLAN 501 and 502 in router mode, but not positive about AP mode. Pretty sure it does as long as you have 386 code on it. But have not tried for myself to confirm. I believe others in here have made use of it in this exact type of setup but can't remember what thread to search for.

Once you have your main wireless up and running it is as simple as enabling the guest wireless 1, rebooting, and going into the CLI and running "ifconfig" to see if you see ones with 501 and 502 listed. Reboot isn't always needed but I've seen my 68U not finish creating all the necessary VLAN stuff without it. Assuming it is there, now you just configure those VLANs on your switch and pfsense and move the stuff you want isolated over to it/them.

VLAN 501 is 2.4ghz guest wireless 1 and VLAN 502 is 5ghz guest wireless 1. I put a tutorial somewhere in here about it a week or so ago.

 

[drinkingbird]

That gives me a much clearer idea as to what to prioritise and what to factor in when planning things. Thank you hugely for taking the time to explain it all - it's so helpful when someone has the depth of knowledge to not only answer questions, but provide answers to the questions I didn't know I needed to ask!

Thank you again!

Having seen a few recent threads on here, it appears that guest network isolation is not available in AP mode, makes sense. Still try it anyway, but if the option isn't there and you don't see any VLAN 501 or 502 in the CLI, you have a couple options:

Get an AP that supports user configurable VLANs (or a second non-VLAN AP and run each one on a separate VLAN from your switch/pfsense), giving you isolation that way without using guest network).

Run Merlin firmware with a script to add VLANs to the AP. There are some examples on here and if you decide to go that route should be pretty easy to put it together.