Expanding Network with VLANS

[RUMC]

Hello. I need some advice about implementing VLANS on the rapidly expanding network at my church. New physical buildings have me trying to meet the needs of the expanding campus. The expansion was largely unforeseen (ye of little faith I guess) so there was no initial plan. So, I have added here and there to try and keep up with the needs, but I realize that isn't the best approach. The schematic shows the hardware I currently have as well as what I would like to maybe try to accomplish to take advantage what I got (most recent addition was managed switches). However, I would not be adverse to purchasing new equipment if need be and provided I could afford it (maybe an er-x router??). I have been reading here for about a week concerning VLANS and implementing them ... but it is difficult to piece all the parts of the puzzle together ... like VLANS on routers in AP mode ... vlan support for the AC86U via vlantcl ... rules for communication between different network parts, etc, etc. Any helpful thought or suggestions?

 

[L&LD]

Given the background given, I would be inclined to go with a commercial or enterprise solution if continued growth is expected. Even if that means having to pay for the initial setup to be up and running immediately too.

@Trip? @coxhaus? Others?

 

[RUMC]

Given the background given, I would be inclined to go with a commercial or enterprise solution if continued growth is expected. Even if that means having to pay for the initial setup to be up and running immediately too.

@Trip? @coxhaus? Others?

Thanks for your response. I have indeed thought about it, but such a solution is far beyond my budget in a place like rural MS. You want injectors changed on your diesel engine ... there are 5 places that can do that very reasonably ... but you better get ready to pay serious money for anything IT. Not that it isn't worth it, but there is just no way we could do that right now. Also, I apologize if I gave the impression we were a colossal church or that growth was exponential (maybe rapid was the wrong word) ... we are certainly not. We are indeed growing, but that growth is measured in years and not in months. However, covid-19 suddenly has more people relying on the church computers/internet since many lack any form of home internet.

 

[L&LD]

In that case, you may want to consider using either the R7000 or the RT-AC66U as the main router and search for solutions for VLANs and robocfg. Unfortunately, the RT-AC86U, while it is the preferred main router of the models you have, doesn't support robocfg and therefore VLANs as easily.

Others will chime in soon. I'm sure you'll have a few suggestions that will work ideally for you.

 

[RUMC]

In that case, you may want to consider using either the R7000 or the RT-AC66U as the main router and search for solutions for VLANs and robocfg. Unfortunately, the RT-AC86U, while it is the preferred main router of the models you have, doesn't support robocfg and therefore VLANs as easily.

Others will chime in soon. I'm sure you'll have a few suggestions that will work ideally for you.

Thanks again. That's a good idea.
I am hopeful considering the collective knowledge of the forum.
In my forum browsing, I did come across this informative post by LeandroBR where he gives details on using vlanctl with the AC86u, but he ultimately edited the post to say normal linux commands can be used and vlantcl isn't even necessary (if hardware switching is disabled).

Hello,

At first, read EDIT 2 (At the end of this post). There is another way to configure VLAN.

# So, the last step, if you want to communicate between LAN interfaces (1 to 4), you must disable HW Switching (this will increase the CPU usage, but my tests showed you can reach 1Gbps with no problem).
# Para funcionar tráfego entre as interfaces físicas (1 a 4), desativar o HW Switching, isto aumentará o uso de CPU, mas de acordo com os meus testes, você consegue atingir 1Gbps sem problemas.
ethswctl -c hw-switching -o disable

EDIT: this last command is necessary, it will increase the CPU usage, but if you keep the hw-switching enabled, the packats will bypass the system processing and it will work as a normal switch, not a managed one.

EDIT 2: I performed some additional tests, after disable hw-switching, it's possible to create VLAN using "normal" Linux commands, so, it's not necessary to use VLANCTL.

 

[coxhaus]

I would look for small business networking equipment that is built for VLANs. 2 VLANs that come to mind are staff and congregation should be separated. There may be others. You will probably want multiple wireless APs to handle the Sunday load.

I like the Cisco small business networking gear and can give recommendations if you are interested. Do you have any networking skills?

 

[Trip]

Thanks for the shout @L&LD

Welcome @RUMC - First, some questions:
I've outlined what appears to be the separate buildings and gear within. Would those be correct? If not, I'd appreciate clarity there.
Also, what exact model PoE+ switches (in red) are those?

As @coxhaus suggested, instead of trying to shoe-horn VLAN capability onto consumer all-in-ones via custom firmwares and scripts, I would replace the all-in-ones with native-VLAN-capable gear -- a wired router connected to the modem (to replace the AC86U, marked in blue), plus purpose-built wifi APs in place of the other all-in-ones (R7000, AC68U and AC66U, marked in orange). Although that sounds expensive, it doesn't have to be. Cisco small business is one option, but there are lower-cost alternatives that could likely work well enough (TP-Link, Ubiquiti, used/refurb enterprise, etc.).

I'll get more into specific replacement gear and a proposed topology once you can confirm the building layout and PoE+ switch models. Thanks!

 

[RUMC]

Thanks for offering to help. I would love to hear all the recommendations and suggestions I can get. Let me see if I can answer your questions.

Do you have any networking skills?

Not a whole lotta skills, but maybe just enough to be dangerous . I have done all the networking thus far by myself while learning a lot along the way (I ran cat6 cabling throughout building from wallplates back to the patch panels, am familiar with shell scripting/CLI/Linux/BSDs, set-up our building automation using Hubitat and different z-wave /zigbee devices). So I am by no means an expert, but I am not totally clueless either. For now, I will be the sole administrator of the network until I can find somebody else that is interested in helping. Nobody else knows or cares what is going on ... they just want internet to work.

I've outlined what appears to be the separate buildings and gear within. Would those be correct? If not, I'd appreciate clarity there.
Also, what exact model PoE+ switches (in red) are those?

Yes, your building segmentation is more or less spot on. Building C is the sanctuary/worship space, Building B is the fellowship/kitchen area, and Building A&D is classrooms/offices. The 4 gigabit PoE+ switches are 2x Edimax GS-1008P, 1x BV-Tech POE-SW802G, and 1x TRENDnet (TPE-TG44G). I used existing cat6 cabling to spread the PoE+ switches around so as to avoid long runs back to one centralized PoE+ switch (suspended ceiling with batt insulation is a headache to get above plus seriously hot in the MS summer).

I certainly wouldn't be averse to replacing hardware. However, it might have to be done in stages as funds become available ... like replace main router (and use existing routers as APs and then replace them as I can). Also, I am certainly open to used/refurb, TP-Link, Ubiquiti, etc. I got the 2x Zyxel managed switches used off eBay for $40 each.

Again. I appreciate the assistance. I am just trying to meet a need and help bring my country church into the 21st century.

 

[coxhaus]

Do any of your current switches support VLANs? I don't want to look them all up. I assume have cable like CAT6 or something connecting the building since there is a black line connecting builds on the diagram. What is it? I hope not wireless. Trying to make a bunch of different brands of VLAN switches talk together is not going to be fun. They will use different terms and do things differently.

Are the POE switches the main wired structure on how the data is passed from building to building? I am trying to understand this statement
" I used existing cat6 cabling to spread the PoE+ switches around so as to avoid long runs back to one centralized PoE+ switch " Are you say you just did not want to string home runs to 1 switch so you are using the router connected wires?
Your diagram shows the black lines connecting the routers. Please explain how the buildings are connected and how many wires there are between buildings? And what type of wire?

My guess is you can replace all the wireless routers with wireless APs and add 1 wired router. Create 2 VLANs one for staff and one for your congregation and see what shakes out. You are currently doing it with 1 LAN. The congregation being only wireless. Use 2 SSIDs with 2 VLANs. I would use Microsoft DHCP by adding another scope for your congregation. This is based on you using your current switches and making them all work with 2 VLANs. You can further divide into more VLANs later but your current network is built for 1 network VLAN.

And of course, I always assign a network to every VLAN. So staff would be the current VLAN and congregation would be the new network VLAN and scope on DHCP.

Your task is to add 1 more network VLAN to all your switches. Leave everything running on VLAN1. Test each switch as we cannot flow the additional VLAN. Then add a DHCP scope to your Microsoft DHCP server. You can do this before you replace any equipment. It does not matter whether you are using DHCP on routers are not. If we have to replace switches then this changes. Make sure if we pull the routers out you still have enough switch ports.

 

[Trip]

any switches support VLANs?

The two ZyXel GS1900-24's are web-managed L2+. The other PoE+ switches are unmanaged, but can potentially be left as-is, as they all feed the same would-be VLAN of devices (IP cams), so aggregate traffic from each could be tagged as a single port-based VLAN on the next upstream managed hop (GS1900 or otherwise).

Trying to make a bunch of different brands of VLAN switches talk together is not going to be fun.

So standardizing on Zyxel GS for managed switching might make sense and also recycle a bit of tech debt.

@RUMC - To confirm, are Buildings A, B, C and D (as I labeled them) actually four separate buildings, or are any of them actually two separate "spaces" inside the same physical structure (A and D, for example)?

 

[coxhaus]

The other PoE+ switches are unmanaged, but can potentially be left as-is, as they all feed the same would-be VLAN of devices (IP cams), so aggregate traffic from each could be tagged as a single port-based VLAN on the next upstream managed hop (GS1900 or otherwise).So standardizing on Zyxel GS for managed switching might make sense and also recycle a bit of tech debt.

We need POE+ VLAN aware switches to run the VLAN aware APs.

The cameras can maybe run off the old POE+ switches if the cameras are wired. But it depends on the AP's switch.

 

[Trip]

We need POE+ VLAN aware switches to run the VLAN aware APs.

Not necessarily. APs could potentially be cabled direct to the GS1900's and run from included injectors. Not quite as elegant 100% managed PoE, but perfectly workable and a potential cost savings, which is a factor here (to a point). We need more info from @RUMC regarding structural limitations, etc. before concluding for sure.

 

[coxhaus]

I don't like it but it could be done. If we add injectors then these VLAN switches are cheap and will do L3 switching.
https://www.amazon.com/dp/B079845S9N/?tag=snbforums-20

He has 2 VLAN switches and four buildings with non-VLAN switches in all 4 buildings. What runs in buildings C and D? I would guess at least cameras and wireless so we need a VLAN presence in each building. Plus he wants to expand in the future to more VLANs like 1 for security cameras.

I would like to know where DHCP runs from. The server only or the router also.

 

[RUMC]

Thanks again for the feedback and suggestions. Let me see if I can address some of the questions:

Do any of your current switches support VLANs? I don't want to look them all up. I assume have cable like CAT6 or something connecting the building since there is a black line connecting builds on the diagram. What is it?

Both of the 24 port switches support VLANs while none of the PoE+ switches do. Cat6 cabling connects everything (black lines in diagram). None of the network backbone depends on wireless connectivity. In fact, I tried to limit the wireless connectivity wherever possible. These are metal buildings (with metal stud walls on the interior which aren't structural) that are basically all connected. Therefore, you can walk from one building to another without ever going outside. Thankfully, this makes it relatively "easy" to run post-construction cabling/electrical/pex plumbing from one building to another.

Are the POE switches the main wired structure on how the data is passed from building to building? I am trying to understand this statement
" I used existing cat6 cabling to spread the PoE+ switches around so as to avoid long runs back to one centralized PoE+ switch " Are you say you just did not want to string home runs to 1 switch so you are using the router connected wires?
Your diagram shows the black lines connecting the routers. Please explain how the buildings are connected and how many wires there are between buildings? And what type of wire?

For instance, I planned for 3 network drops in the Pastor's Office (in Building D) and pulled 3x Cat6 cables running directly from the drops back to the patch panel and switch (in Building A). When we decided to install IP Cams (which was indeed an afterthought), I decided to take one of those 3 cables and instead feed an AP (AC68U) which feeds a PoE+ switch in order to supply connectivity to that side of the building. Now the 6x IP Cams in that vicinity can simply run back to that area's PoE+ switch rather than having to run 6x new Cat6 cables all the way back to the PoE+ switch in Building A. I still got 3 drops in the Pastor's Office (the 3rd drop comes direclty from a port on the AP).
[/QUOTE]

To confirm, are Buildings A, B, C and D (as I labeled them) actually four separate buildings, or are any of them actually two separate "spaces" inside the same physical structure (A and D, for example)?

They are all really separate "spaces" (Fellowship Area, Sanctuary, Offices, Classrooms, Storage, etc) within connected buildings. They were constructed as separate buildings at separate times but are connected (didn't want people out in the weather navigating from one space to another). Building A and D are indeed actually within the same physical building. If one were to look at it, one would think it is actually one big building built all at the same time ... which was our intent when we added on new buildings.

What runs in buildings C and D?
I would like to know where DHCP runs from. The server only or the router also.

Drops in Building C (along with AC66U AP Point) are feed via Cat6 cabling from the switch in Building B. IP Cams in that space will connect to Building C PoE+ switch.
Drops in Building D (along with AC68U AP Point), with the exception of 3rd drop in Pastor's Office (see above), are feed via Cat6 cabling from the switch in Building A. IP Cams in that space will connect to Building D PoE+ switch.
Right now, all DHCP is handled by the main AC86U router. Got Windows Server up and connected and ready to go ... just delayed deploying it until I get all this figured out

 

[coxhaus]

You will want to run DHCP and DNS from the Microsoft server to keep active directly working well.

Can you figure out how to run one of your switches as a layer 3 switch? I just do Cisco.

 

[Trip]

@RUMC - Thanks for the additional feedback. The layout below (click for full size) makes maximum use of what you already have, adds as little cost and complexity as possible and also positions for future upgrades. Items in black stay the same. Items in green are new or optional.


Physically Unchanged: AT&T modem, GS1900 switches, PoE+ switches, IP cams and host connections (server, NAS's, PC, etc.)

Physical Changes (~$300 of gear, <$100 of Cat6 + connectors):

1) Router Swap -- The AC86U (R1) is replaced with a VLAN-capable wired router, specifically one offering SQM to eliminate bufferbloat on the 10/1 DSL link. I'd recommend a $60 Ubiquiti ER-X, running either native Smart Queue QoS (fq_codel + HTB) or a back-port of CAKE, tuned appropriately.

2) Topology Changes -- R1 should be wired to the GS1900 core switch (S1), which is then backboned to the other GS1900 (S2). Both then downlink to their respective PoE+ access switches. Based on current cabling constraints, this gives you the best traffic flow and lowest broadcast overhead.

3) Wifi APs - Same-brand, VLAN-capable APs ("AP1" through "AP4") will replace each of the consumer routers (86U, R7000, 68U, 66U). For each AP I would run a new Cat6 home-run to the closest GS1900 (S1 or S2), for dedicated backhaul, plus power via PoE injectors, instead of having to buy managed PoE+ switches to replace the current ones (which are just serving IP Cams anyways, so all that traffic can be "unmanaged" and tagged as one VLAN on ingress into the GS1900's). I recommend TP-Link Omada EAP225v3's, which are a mere $60 each and come with PoE injectors included (~$240 total), plus you can run the Omada controller (necessary for central admin, seamless roaming and guest portal) for free on the Windows server.

4) Backbone Upgrade (optional) -- Since you have core network services (DHCP, DNS, NAT, etc.) sitting on either side of S1 and S2, you might consider an additional run of Cat6 between the switches to form a 2-port LAG. This would add redundancy and 2x throughput. Alternatively, you could run fiber between the two switches (the GS1900's have 2 SFP ports), which, although adding no more bandwidth now, would drop latency between S1 and S2 to almost zero, and provide 10/40Gb backbone when the time comes for switch upgrades.

5) Server/NAS Relocation (optional) - Additional to or instead of a backbone upgrade, you might also consider moving the server and/or NAS 2 to Building B, and consolidating your "data center" there, as it's usually best to connect as many core network services to your core switch, whenever possible. Alternatively, you could cable R1 to S2, making that your core switch, and bring NAS 1 to Building A. Or if each NAS serves mostly hosts in the same building, you could leave as-is.

Config Changes:

R1 - Will need interfaces, VLANs and DNS and DHCP forwarding (to the Win server) setup for each respective subnet (VLAN) you wish to setup, as well as firewall rules created to drop/permit traffic between VLANs (mostly drop, based on your requirements).

S1 and S2 - Will need identical VLANs defined, plus corresponding netmask, gateway and DNS, as well as the correct assignments of untagged and tagged VLANs to ports.

APs - Will need SSIDs and SSID-to-VLAN mappings -- doable centrally from the controller.

Windows Server - Presuming you do DHCP on the server, you'll need a DHCP scope for each VLAN. You might also consider running DNS on the server as well, depending on how much name-based resourcing you plan on hosting in your Windows environment.

Looking forward:

Layer-3 Core Switch - As long as you don't foresee super heavy local static or inter-VLAN routing, doing layer 3 on the gateway should be fine for the near-term future. An L3 switch would make certain local behavior faster, but at your average throughput it's likely peanuts compared to lower hanging fruit (layer 2, wireless access, etc.). You can always upgrade to a layer-3 core switch later, without having to undo anything major.

Fewer, Bigger Managed PoE Switches - This comes down to cost savings and cabling constraints. For now, injectors off the GS1900's will work. If/when you see any bottlenecks in your access uplinks, you can then build a case for higher-density managed PoE switches, plus more Cat6 home-runs. Either way, not a show-stopper for now.

-------------------
Hope that helps give good guidance for now. Feel free to ask questions as needed.

 

[coxhaus]

It looks pretty good trip but I question 3 links between 1900 switches. And fiber is a nice touch. My only problem with his setup is the server and router are on opposite switches so you will have a lot of traffic going back and forth between switches. Your core switch ends up in building B. If you are going to run fiber at 10 gig then the issue goes away. My other thought if you run a second CAT6 cable between building A & B then you can hang your router off building's A switch making it the core L3 switch. I like having the server in the core switch. I would also think moving the NAS 1 to where all the other servers are would make more sense

Buildings C & D either need a second CAT6 cable or a VLAN switch.

I would not buy TP-Link as their software support is too short term.

 

[Trip]

Thanks cox. I should have added, the proposed links between switches are really an either/or proposition, not both; two copper ports in LAG for now, then sub out both for a fiber link later, or just do a run of fiber straight away.

And yes I forgot to mention getting the server attached to the core, so either moving it to Building B, so it sits on S1, or cabling R1 to S2 (instead of S1) or just running fiber between S1 and S2. (I've now edited my original reply to included a consideration for moving all "data center" and network services hardware into the same building as whichever switch acts as the core).

I suggested TP-Link APs mainly on cost savings, less so support longevity. For that, Cisco CBW140AC's would likely be better, but at roughly 2x the cost per unit. The OP will have to decide what he places more value on.

 

[coxhaus]

So now the work begins. Learn VLANs and how to set them up on your equipment.

Once the network structure is in place then it should be a simple thing to add more VLANs probably less than 30 minutes to add a VLAN.

 

[RUMC]

Thanks so much for the clear and complete, yet concise, advice ... this is exactly what I needed. I have looked over the comments/schematic many times to let it sink in, and I do have a few questions (which may seem like statements but are for just me trying to make sure I am understanding properly):

1: Connecting S1 and S2 with a fiber optic cable is certainly doable and a great idea which I didn't know was even possible. I could then using the existing Cat6 cable to connect R1 to S2 as suggested. However, I get a little confused by the terminology (SFP, LC, OM1/2/3, etc) of fiber optic cables (as I have never dealt with them) and making sure I get the appropriate cable I need with ends that would work with the Zyxel switches.

2: I can "get away with" unmanaged PoE+ switches only because all traffic connected to them will belong to the same VLAN (back at S1/S2). Similarly, this is the reason the APs could not be connected to the existing PoE switches ... APs will be different VLAN from IP Cams. Therefore, PoE injectors must be used to power the APs which have to be connected back to S1/S2. The only way around this is managed PoE switches.

3: Likely the next addition (maybe in a few years) will be a Family Life Center (gym) which will indeed be a separate building but still connected by a hallway. For that, I could come off S2 using fiber optic cable to a managed switch in the FLC and then run APs and/or PoE+ switches (for IP Cams) from it.

4: If I somehow could manage to upgrade to L3 switches (likely used from eBay) ... what would be the change/benefit? The main inter-VLAN communication I can foresee would be computers/wireless clients viewing the IP Cams: maybe the computer in the Pastor's Office or a wireless church tablet we give to parents during the service so they can view their child via the IP Cam in the nursery, or a security team member monitoring the cams from a tablet during the services, etc.

5: Just an observation: I used to complain about our 10/1 connection at the church (which works via a "fixed wireless" antenna to an ATT cell tower) but through coivd-19 I have become painfully aware of many, many, many families that don't even have that which in turn has made distance learning, working from home, and those related decisions nearly impossible and difficult. So, I will not complain anymore except to say I wish some entity would come along to offer those families something. High speed internet is becoming almost an essential utility like electricity.

Again, many thanks for your time and assistance.