Expanding Network with VLANS

[Trip]

@RUMC - You're very welcome. Responses to your numeric bullets below:

1) Indeed, certainly doable, and in fact, that may be the optimal setup: fiber between S1 and S2, and make S2 your core switch by cabling it to R1, instead of S1. Regarding the terminology and components behind fiber, here are guides on fiber types (single vs. multi-mode), connector types and SFP transceivers. Long story short, for your switch-to-switch run, you'll most likely end up using an LC-type connector and either single-mode ("OS"+#, usually OS2) and LX transceivers, or multi-mode ("OM"+#, usually OM2 or OM3) and SX transceivers. The easiest practical way to get a ready-to-connect fiber run is to buy pre-terminated fiber patch at the length you've measured (plus 5-15 foot service loops on either end), with connectors already fastened. This way, all you have to do is lay the fiber run, plug in your compatible SFP transceivers into the switches, plug in the connectors at each end of the cable into the SFPs, and you're done. Also, if you don't already know, when handling fiber, want to pay special attention to not over-bending it, or bending it under its minimum bend radius, as the glass can and likely will break. If you sense the pull path may compromise the fiber too much, you can also look into bend-insensitive and/or armored fiber patch. If you want help finding links to such products, just say so and I can do some digging for you.

2) Correct on all points.

3) Correct again. Depending on if you can home-run all IP cams, APs and keystone jacks, you might consider just running a single, higher-density managed PoE switch (instead of a managed, non-PoE switch plus smaller PoE switch combo), to keep your network segment there as flat and easily-managed as possible. You might also consider consolidating to fewer, higher-density PoE managed switches in buildings A, B and C at some point in the future, provided you can home-run all IP cams to that switch at some point. The combination of doing all of this. If you do all of the above, you might then consider upgrading the GS1900's in buildings A and B to the same, newer model of switch going into the gym, and choosing a stackable model, whereby all three can form a switch stack, which acts as a virtually-combined single backplane and can be managed from a single IP as such. A very nice thing to have in a distributed, "ring" type topology such as yours.

4) The main advantage is that you'd be offloading all of the local layer-3 traffic processing off of R1 (your gateway/router), freeing it up to just perform NAT and other gateway-only services (VPN, possibly DNS if the Win server isn't doing DNS, etc.). Doing local routing on the switch will also make traffic route and resolve a bit faster (in microseconds, perhaps even low milliseconds) when things like DHCP queries, services forwarding and/or local routing takes place. Again, for a network your size and traffic load, I'd call this trivial at best for the time being, and if your network load grows, something you can work in later.

5) A very humble take-away, indeed. And I stand corrected on the internet type: microwave/cell, not DSL (FYI, having SQM on your router will help just the same). Hopefully your area will get better connectivity soon.

One side tid-bit: I presume you're either in Richton or Raymond (only so many RUMC's in MS)? No need to divulge if you prefer not to.

Hope that helps again.

 

[coxhaus]

If you can't do 10 gig with fiber in your switches then say don't bother with fiber and pull another CAT6 cable so the router can run off building's A switch. Set the switch up for inter-VLAN routing with static routes to the router. Run Building's B switch as layer 2 on a trunk port from building A. I thought your current switches are rated 2+? I know nothing of your current switches. I will say fiber is safer going building to building during lightning storms. But where do you stop with the expense. I have a couple of threads on here on how to setup a Cisco small business switch in layer 3 mode.

I personally would not use dumb switches to pass VLANs for a number of reasons being they are not rated to do that and there is no security. Pull another CAT6 cable to make up for the lack of VLAN switches in buildings C&D.

 

[RUMC]

I hope y'all had an enjoyable labor day. Thanks again for the prompt responses ... I have studied over them and the various links ... very helpful. So I have started to put together my to-do list:

Purchases:
A: Purchase VLAN aware router (Ubiquiti ER-X): $60
Any advantage to this version with an SFP port for $99?

B: Purchase 4x APs (TP-Link Omada EAP225v3): $240

C: Purchase pre-terminated (with LC terminations) OM2 fiber optic cable + 2x SX transceivers (should be good for 10Gb): $90
The SFP slots on my GS1900-24 switches appear to be 1Gb from the specs I found. So, to @coxhaus's point, do I need fiber or another run of Cat6 for the 2-port LAG? I guess the 10Gb fiber would help future-proof the network but with an associated price.

Actions:
1: (Re)Locate new router (+ ATT modem + possibly NAS 1) in Building A and attach directly to S2.
I suspect more traffic consistently on S2 (except for Sunday mornings when S1 traffic will likely increase - we stream services).
On 2nd thought, I may have to locate the ER-X in Building B at the current location of the ATT Modem since that is where cabling from outside antenna enters the building.

2: Connect S1 and S2 via fiber or 2x Cat6 runs for LAG.

3: Run Cat6 as needed to Building C and D to connect unmanaged PoE+ switches and APs back to S1/S2.

4: Install IP Cams and connect back to that building's respective PoE+ switch.

5: Install APs.

6: Learn VLANs and how to set them up on my equipment.

7: Sale AC66U, AC68U, AC86U, and R7000 wireless routers. (Thanks Merlin for your awesome firmware and the SNB community for all the great tools)
However, I might could use them for networking at smaller area churches that are struggling to stream services.

Did I leave anything out? Again, many thanks for all the assistance.

One side tid-bit: I presume you're either in Richton or Raymond (only so many RUMC's in MS)? No need to divulge if you prefer not to.

True and solid guesses. However, the "R" doesn't refer to the city name or we would be LUMC
Richton and Raymond are both nice areas of the state ... I guess except for the heat/humidity this summer which has been absolutely brutal.

 

[coxhaus]

You are going to find more than likely fiber for 1 gig will not work for 10 gig so there is no future in using 1 gig fiber at this late date. There is no standard for fiber and the fiber diameter is smaller for single mode 10 gig than multi-mode 1 gig fiber. 1 gig fiber is past it's time. Sounds like your switches will not support 10 gig then use 2 CAT6 cables as I describe above.

I would stay away from TP-Link but there are people which install them. Just make sure you quit using TP-Link's wireless hardware when they don't write patches for it any more. TP-Link will just leave you hanging. TP-Link will not declare it EOL. You will need to keep up with patches for TP-Link. I use Cisco's WAP581 wireless APs. I am a Cisco guy. Cisco will support thier hardware for many years. I have seen WAP581 wireless APs on eBay for a $100 used. Ruckus is another wireless AP recommended on this site that I think would be better than TP-Link. I have not used Ruckus wireless APs so others would need to fill in.

Usually with commercial buildings the Dmark is set and cannot be moved. AT&T can only be installed at the Dmark on commercial buildings. This is why I said we need an additional CAT6 cable from building A to B so the router can be hung off of the switch in building A which will make a better core switch based on your design.

 

[Trip]

@RUMC - Very welcome. Answers and comments below:

Re- the ER-X-SFP, routing power and bus speeds are the same (comparison), so no real benefit there. You could save one copper port on both the router and your core switch by using an SFP-based DAC cable (presuming the switch is close enough, so probably to S1 only), but you wouldn't gain any performance. So I'd stick with the ER-X. If you ever do need SFP on the router, it probably won't be until you'd want to swap to a higher-power platform altogether, at which point, you may want an SFP+ 10Gb interface anyways, for WANs over 1Gb.

Regarding fiber for the LAN backbone, you'd need a switch upgrade to to make it worth it, and I would also argue for having PoE on-board and home-running all your IP cams to those switches. Considering the cabling work required, the switches themselves, which are still $250+ each new, $100+ used (ex: Aruba S3500's @ $130), plus SFP+ transceivers (~$20 each), and OM3 or higher-grade multi-mode to realize 10Gb, all in you'd be looking at ~$1000 at minimum. Likely way beyond even a question of cost-vs-benefit. So for now, the best option would be to relocate the Windows server to Building B if possible; that way you'd only need one additional Cat6 run for a 2-port LAG. If you kept the server where it is, you'd need a new run from S2 to R1, plus another between S2 and S1 for a two-port LAG.

All your other steps are pretty much spot-on.

Regarding the unmanaged PoE+ switches and VLAN traffic, there is a small chance they may not properly reflect the VLAN-tagged packets (from the GS1900's) back to the IP cams, in which case you may need to replace with managed variants. TP-Link or Netgear would be minimum-viable options that aren't too costly. I've found Cisco SG to be even better in a mixed-vendor stack (historically) as well as supported for longer, but the added cost is there, too.

Hope that helps again.

 

[coxhaus]

Building A sounds like more the office area. I would keep the server there and make the wire work for you. You do not have to have a LAG between switches. It would be better and it is just as easy to pull 2 cables as 1. How many devices are hung off the switch in Building B and what are they besides cameras and wireless?

Your wireless is going to be mainly internet traffic so you will be limited by your internet speed. What is your internet speed?

Once you move the NAS traffic may be low. If you have a lot of PCs using the NAS then maybe we need to rethink this. You are doing this with a bunch of little routers so I think we are making a big improvement with this design.

 

[Trip]

What is your internet speed?

Internet is 10/1 Mb AT&T microwave wireless. A limitation, but de-bloatable with SQM at least.

 

[coxhaus]

Wow that is terrible. They need faster internet or on Sunday the system will not handle all the wireless connections.

No wonder they were not complaining about all those little routers.

The microwave wireless I have seen is high latency on top of the small bandwidth.

I definitely would not worry about fiber and hope for faster internet. Doesn't AT&T provide 25/1? It would help a little. I have a friend with AT&T 25/1 and caps so he cannot run movies.

 

[RUMC]

Thanks again @coxhaus and @Trip for the guidance.

How many devices are hung off the switch in Building B and what are they besides cameras and wireless?

Most are drops to different areas of that Building. However, not all drops are currently used.
We have (all wired) on S1: an Epson projector, sound booth PC (which we live stream from on Sundays), 3 others workstations, 2 printers, Smart TV, NAS 1
Building A (with S2) is definitely more of an office/classroom building and will typically have more traffic.

Wow that is terrible. They need faster internet or on Sunday the system will not handle all the wireless connections.

Exactly, we pray before service every Sunday that the stream will go out successfully with no problems . However, 10/1 is what they "guarantee" for ATT "fixed wireless" but (thankfully) due to the church's rather close proximity to the tower we usually see 25-30 Mbps down and 2-3 Mbps up. There is a possibility that our local power company may start offering high-speed internet. Also, I saw the following news recently of an agreement between "MS Public Service Commission, Mississippi Power and C Spire that will increase access to broadband and expand economic development opportunities. This project will create a "fiber ring" bringing broadband access to 6,500 locations" in our area. I think covid-19 has finally made the right people painfully aware of the lack of high speed internet in rural MS.

Regarding the unmanaged PoE+ switches and VLAN traffic, there is a small chance they may not properly reflect the VLAN-tagged packets (from the GS1900's) back to the IP cams, in which case you may need to replace with managed variants. TP-Link or Netgear would be minimum-viable options that aren't too costly.

I was just pricing managed PoE+ switches (especially for Buildings C and D).
Maybe this is a stupid question but .... would my network (and maybe my wallet) be better off purchasing a couple used Layer 3 Cisco 48 port PoE swtiches (like these WS-C3560X-48P-S here on eBay) for Buildings A and B, connect them together via a 2-port LAG, and run everything directly from them? I know a great deal of cabling would be involved, but as @coxhaus reminded me ...

it is just as easy to pull 2 cables as 1.

 

[coxhaus]

25-30 Mbps sounds a lot better than 10.

Cisco enterprise switches like the WS-C3560X-48P are going to be better quality switches than the small business gear switches. They will work in a much bigger network and not be lacking like small business switches. The bad. You have to pay for IOS on the switch. Most people buy maintenance so you get free IOS upgrades. This will cost a lot. The other bad thing is enterprise switches are built for wiring closets so they are real loud. You will have to configure with command line. The WS-C3560X-48P will go EOL, end of life, next October so you will need to replace them real soon. And really at this late date these switches will probably not have any more IOS updates. Cisco quit selling WS-C3560X-48P switches in 2016. It is not something that would benefit your network. So I recommend against buying these old Cisco switches.

With your slow internet I am not sure a switch upgrade will help your network.

 

[RUMC]

25-30 Mbps sounds a lot better than 10.

Cisco enterprise switches like the WS-C3560X-48P are going to be better quality switches than the small business gear switches. They will work in a much bigger network and not be lacking like small business switches. The bad. You have to pay for IOS on the switch. Most people buy maintenance so you get free IOS upgrades. This will cost a lot. The other bad thing is enterprise switches are built for wiring closets so they are real loud. You will have to configure with command line. The WS-C3560X-48P will go EOL, end of life, next October so you will need to replace them real soon. And really at this late date these switches will probably not have any more IOS updates. Cisco quit selling WS-C3560X-48P switches in 2016. It is not something that would benefit your network. So I recommend against buying these old Cisco switches.

With your slow internet I am not sure a switch upgrade will help your network.

That is what I needed to know. Thanks for the answer.

 

[Trip]

Maybe this is a stupid question but .... would my network (and maybe my wallet) be better off purchasing a couple used Layer 3 Cisco 48 port PoE swtiches (like these WS-C3560X-48P-S here on eBay) for Buildings A and B, connect them together via a 2-port LAG, and run everything directly from them? I know a great deal of cabling would be involved, but as @coxhaus reminded me ...

Absolutely not a stupid question, by any means. I avoided going there earlier due to simply presuming you wouldn't want or be able to home-run all IP cams to the two main switches, but if you are willing to entertain that, then I definitely would consider converging to fewer, higher-density, managed PoE switches. An adjusted layout would look something like this (click for full-size image):

​

Obviously, a lot more wiring work (in green), but you make the network a lot more efficient with fewer, higher-capacity backplanes and way fewer cascading points of failure. As @coxhaus detailed, though, you'd have to consider the implications of the chosen switching hardware. Cisco Catalyst is indeed robust, both in its code and its physical build, but especially for older 24 and 48-port PoE units, you have to account for noise and heat, plus configuration (IOS command line required) and licensing/support, should you choose to add it. That doesn't mean you shouldn't consider a 48-port PoE solution. Other brands/models could be a better fit, such as HPE Aruba/ ProCurve (refurb 25xx or 29xx series) and run at a potentially quiet-enough levels. There are also small-business, GUI-centric options, which include support and run quieter/cooler, such as Zyxel's GS1920-48HPv2, which runs at a class-leading 26dB (quieter than a whisper), but it is $499 new, so you'd be back to a $1K spend. Cisco also has some new CBS250/350 series switches coming out to replace the SG250/350 series, but even the 250 models are roughly twice as costly as the Zyxels.

So it's still a novel idea; it just may take some creative sourcing and/or a bit of learning on your part to get configs built and running if you go the used enterprise route. Either way, certainly worth exploring. As you said, it may save you considerable opportunity cost in the long run.

 

[coxhaus]

Once you run the cable for the router you can locate the modem and router in the office where the server is if it is more convenient or the router any way depending on the cable into the modem. Your choice.

 

[RUMC]

Many Thanks.

View attachment 26131​

Would I still need the PoE injectors for the APs if I feed them from a managed PoE Switch?

Other brands/models could be a better fit, such as HPE Aruba/ ProCurve (refurb 25xx or 29xx series) and run at a potentially quiet-enough levels. There are also small-business, GUI-centric options, which include support and run quieter/cooler, such as Zyxel's GS1920-48HPv2, which runs at a class-leading 26dB (quieter than a whisper), but it is $499 new, so you'd be back to a $1K spend. Cisco also has some new CBS250/350 series switches coming out to replace the SG250/350 series, but even the 250 models are roughly twice as costly as the Zyxels.

That sounds good as there is plenty of reasonably priced used equipment on eBay ... will start looking.

 

[coxhaus]

You do not need POE injectors if your switch has POE+ power. Why do you want a switch upgrade?

If you are going to buy a POE+ switch make sure you buy 802.11at not 802.11af. af is old and dead. All the new devices require more power.

I run my core switch and my POE+ as separate switches because the big POE+ switches are loud. I have a Cisco SG350X-24 which I am looking for a 2.5 gig with 10 gig uplink POE+ switch. My Cisco WAP581 wireless APs have 2.5 gig ports on them and I would like to switch from 1 gig to 2.5 gig on my WAP581 wireless APs. The switches are still pricey so I am still waiting.

 

[RUMC]

You do not need POE injectors if your switch has POE+ power. Why do you want a switch upgrade?

If you are going to buy a POE+ switch make sure you buy 802.11at not 802.11af. af is old and dead. All the new devices require more power.

I run my core switch and my POE+ as separate switches because the big POE+ switches are loud. I have a Cisco SG350X-24 which I am looking for a 2.5 gig with 10 gig uplink POE+ switch. My Cisco WAP581 wireless APs have 2.5 gig ports on them and I would like to switch from 1 gig to 2.5 gig on my WAP581 wireless APs. The switches are still pricey so I am still waiting.

So for now I would be better to keep S1 and S2 and the PoE+ switches separate? Should I consider managed PoE+ switches?
I was heeding @Trip's advice:

... then I definitely would consider converging to fewer, higher-density, managed PoE switches.

 

[coxhaus]

You need to go ahead and build your network. Don't get caught up in the upgrade mode. You can always upgrade. We gave you a plan now go and make it work.

 

[Trip]

For now, to keep cost down, I would stick more to the first plan. Injectors + APs off of the main GS1900's; leave the unmanaged switches as-is and see if they'll forward returning tagged traffic to the IP cams. For AP runs, just make sure you leave enough service loop at the switch end that you can wire directly into a PoE switch if you ever replace the GS1900's with something with PoE on-board. Beyond this first batch of changes, before going further on the internal network, I would focus on upgrading internet in any way possible. That 10/1 is brutally slow; even the fanciest QoS schema in the world can only do so much with 11Mb aggregate...

Looking beyond that, though, as I stated earlier, everything you're doing now can serve as the basis towards more upgrades, with hardly any back-tracking (if any at all). You can just add the upgrades over time as opportunity presents (for example, gradually subbing out the unmanaged switches for managed ones, then perhaps going a layer beyond if/when you can afford new distribution/core switches, and home-running anything that isn't, like the IP cams).