The move back to OpnSense, 5 Pillars to build network on.

Maverick009

Regular Contributor
I have previously discussed, my many network changes and test, including a most recent discussion of moving from Opnsense back to Pfsense and my reasoning why. Now fast forward to 2021 and further testing, including with the latest major release of Opnsense 21.1. Initially my logic for going back to Pfsense was due to a little more stability with some features/drivers and the Shaper wizard was a blessing. It also did not mean I would stop testing software/hardware and features out. So what did I do? I for one read more about the differences, and further educated me on the tech and differences between them. The next thing was to plan out what I really wanted with my network and what would be connected.

There was 5 pillars I built my network on:

1. Security

2. Updatable

3. Stability

4. Performance

5. Expandability and Flexibility

Let me explain the 5 pillars further, to give a deeper understanding. Security was the first pillar and sits at the top, as no matter how fast or updatable your network can be, it is nothing if it not secure. With my choosing of Opnsense, it gives me more then enough security, and even goes as far as weekly updates for security fixes or updates. It also goes hand in hand with the 2nd pillar, which was to be updatable. Opnsense, not only gets the weekly security updates, but also gets point releases that enhance or patch the current major release you are on. Speaking of the major updates, you can expect a major update roughly every 6 months, and the latest version 21.1 AKA Marvelous Meerkat did some major improvements on the drivers and stability enhancements, which is very welcome, along with supporting newer NICs including the 2500Mbps cards that have been flooding the market in the last year as the new standard and with Opnsense, you can expect to get all major updates, and the weekly and point releases with no catch. The latest irritation of Opnsense (21.1.3 as of this writing) has been rock stable and performs very well. As far as performance goes, I currently have a Netgear CM1200 Cable Modem plugged into the Opnsense Firewall router with a single 1Gbps connection on both ends, and running speed test, all my devices are able to hit the full bandwidth of the Gigabit internet speed (940Mbps due to overhead with Comcast), and no errors or issues. I also have full intranet performance too, and the Opnsense Router keeps up with my network's needs. Now the last pillar, is partially a need but also more of a want and I can say if you go completely the custom route, you can achieve this goal very easily. The hardware in place will give me the expandability as I need it and the flexibility to work with what I have and what I may want/need down the road.

Now that I have explained the 5 pillars, lets, dig further into the last 3, as the first 2 already come by design. As I mentioned, stability is a big part of the network, and the latest version of Opnsense, went a long way to improving the framework, and driver stability. Now I am not only looking for stability in software alone, but also in hardware. Right now I have Opnsense 21.1.3 running on a custom rackmount system running, Intel Core Q6600 2.4Ghz Quad-Core CPU on a Gigabyte G41MT-USB3 Motherboard, with 4GB Corsair DDR3 Dominator Dual-Channel Memory. The OS is installed on 240GB SATA SSD. As far as NICs go right now, I have the 1Gbps Realtek card built-in to the motherboard, an Intel I350 Quad Gigabit PCIe x4 card, and a Dual 2.5Gbps Realtek powered PCIe x1 card. Originally I was bridging 2 ports of the Intel I350 card with at least 1 port on the Realtek Dual 2.5G card, but have recently forgone bridging to take my network further into isolation and a little bit more performance. It also allowed me to experiment with separate subnets too. I now have the Intel I350 Quad NIC being used with 1 port as a WAN port from the cable modem with the 2 port above reserved for LAGG (The modem supports Link Aggregation to multigig speeds and If I go this route, on my Gigabit connection, I can get about 1.1-1.25Gbps as Comcast leaves some headroom). The other two ports of the I350, are plugged into a TP-Link T1600G-28TS Managed Switch, as I am looking at possibly separating some of the ports on different subnets as well. The 1 port of my Dual 2.5G card is connected currently directly to my Main gaming/multimedia custom computer's 10G NIC and achieving the full 2.5Gbps speed on a different subnet from the rest of the network.

The Network is pretty stable, and I am still achieving full performance even with splitting the network subnets on the current hardware I have. The setup software wise, can be a little confusing, but if you have a basic understanding of networking, it can be fairly easy to get most of the settings right, The hardest part more or less is the Firewall rules configuration, especially when working with more then one LAN, but there is guides and assistance you can find just by searching or going to the forums. I think the hardest part was getting VLAN to work, and that so far I could not get configured correctly on the Opnsense side, but as I mentioned, I have a connection to a managed switch which does support out of the box VLAN and may be easier to just let it manage VLAN connections. Work in progress is what I will say. Overall I have two different subnets at this moment and with the firewall rules, each subnet can connect to the internet and also speak to each other, for instance, I can type the IP address of a device on the one subnet from my main system that is on a different subnet, and it just works.

I also have an Asus GT-AX11000 Router, which was considered the highest home wireless router you could by at the time, connected to network in Access Point Mode. It is connected to the Managed Switch with two 1Gps Ethernet connections in LAGG giving essentially a 2Gps connection and eliminating any major bottlenecks. From that I have all my IOT devices connected on the 2.4Ghz radio, and have the 5Ghz radios split handling various high performance devices, and smartphones. I also have the GT-AX11000 running in AiMesh mode with an Asus RT-AC3100 Wireless router as an AiNode wired into the network through the Switch and managing, the Living Room Sony 4k 65in Android TV, 4K Apple TV, Sony Soundbar and Sony PS5 connected wired with the Xbox Series X wirelessly connected. It also helps expand the wireless connection out, and all IP addresses are managed from Opnsense. There is no performance penalty ether for how I am using AiMesh, and I can rely on Opnsense being the sole Firewall hardware piece of the network.

Using this latest version of Opnsense and now getting past testing it in a production environment, I can say safely that Opnsense may be the better option long-term, as they are redoing the framework of the OS, instead of directly just being a semi copy of Pfsense, and embracing newer tech while at it. I also learned that although the Shaper, does not currently have a wizard, it was due to re-writing the Shaper, and from the forums and searching, it looks like a wizard will be added at a later date. In the mean time, I found some tips and tricks to help me add some shaping settings to my network to manage QOS. With Pfsense pulling back on some features, requiring you to buy their hardware/partner hardware to get faster updates and new features, or forever wait until they push it to the free version, just did not sit as well with me, plus they are going between 1-3yrs between major updates, just doesn't feel like they will be able to keep up as networking is changing quickly and possibly much more then say 2yrs ago due to this pandemic creating a new need, pushing people from smart devices, back to desktops and laptops and network to support them. I still have the expandability, security, and get updates on a regular basis with major releases at least once a year, making sure my network stays secure and can support my needs now and my wants and needs later. I am also planning a major firewall router upgrade moving to at least a 6 Core/12 Thread AMD APU and 16GB DDR, but Once I am ready for that and can secure the hardware, I will share my experience then as that will fit my last pillar much more.

I hope this write up gave you more insite and why even planning can be a good thing no matter the network size. The biggest thing I hear is home router running stock or 3rd party vs. Opnsense/Pfsense as the software is hard to configure. Yes the deeper you go into the software, Opnsense can become more complicate, but out of the box, it will assist in configuring a WAN and LAN connection for basic usage and set the default rules. It is when you go beyond that or want custom rules, and even then just a little reading or search on the internet can assist in most cases. Also with Opnsene, there really is no hardware limit to how many clients can be connected, as consumer home routers, can be limited in addresses and/or hardware functionality. Just keep that in mind when planning your network for now and expandability.
 

avtella

Very Senior Member
Glad you got your setup working the way you want.

Just want to clear up a misguided statement, pfSense Plus will not require hardware purchase to get faster updates or additional features. They have already said Plus would be free for third party users and hardware before June in their initial statement on their blog. Only difference is Plus will be more closed off with certain additions and features that would slowly trickle into the open source version, my guess is this is to prevent the likes of OPNSense from using or implementing those additional features especially after the very sad childish back and forth between these two dev groups.

Netgate also does put a lot back into FreeBSD both financially and dev work wise, for example including sponsoring work for WireGuard integration a lot of which OPNSense also benefits from. I think they feel they need to monetize for their contributions and hence the Plus distro.

Still some key parts like pfBlockerng (pfSense), Sensei (OPNSense), WireGuard or QoS setup that are huge differentiators between the two.

Additionally some have used used two VMs with OpenWRT for the Cake SQM along with OPN or pfSense on the side for the Firewall. FreeBSD still is a bit behind QoS wise I feel vs Linux (OpenWRT).

Lastly for hardware sizing for OPN/pfSense, a low wattage 2/4 Core CXXX series Atom with 4-8 GB RAM would suffice for most easily hitting gigabit. And if doing a lot of VPNs and IDS/IPS a Quad Core i3/Xeon D or Ryzen/Epyc Embedded. Hardware especially used can be found for like $200-400 even those with an open PCIE slot. I’d still recommend an Intel/Chelsio/Mellanox Nic over any Realtek chips as those aside from issues with hardware offloads (even today) also use more CPU in general.
 
Last edited:

Maverick009

Regular Contributor
Glad you got your setup working the way you want.

Just want to clear up a false statement, pfSense will not require hardware purchase to get faster updates. They have already said plus would be free for third party users and hardware before June in their initial statement on their blog. Only difference is Plus will be more closed off with certain additions and features that would slowly trickle into the open source version, my guess is this is to prevent the likes of OPNSense from using or implementing those additional features especially after the very sad childish back and forth between these two dev groups.

Netgate also does put a lot back into FreeBSD both financially and dev work wise, for example including sponsoring work for WireGuard integration a lot of which OPNSense also benefits from. I think they feel they need to monetize for their contributions and hence the Plus distro.

Still some key parts like pfBlockerng, Sensei or QoS setup that are huge differentiators between the two.
I miss read it some, but not completely. In the short term they will require hardware purchases to get that version of pfsense and looks like some previously purchased hardware that had the old FE edition will get upgraded. They are hoping to make a standalone version of the plus edition available by end of June 2021 if not sooner with some locks to the licensing. Basically Home and Lab custom built will be ok, but no commercial usage. With the addition, looks like they will also try and match Opnsense to an extent on faster updates. They do put one big coveat in their announcement, and that is that the free CE edition will get slower updates, and they will evaluate the need weather they need to speed up updates, or possibly abandon altogether based on how you read it.

I do agree that a lot of this stems from Pfsense stopping others from copying and vice versa as they seem to not like competition. This latest edition from what I can see is more or less a money grab and adding locked code into the OS. They are both pretty much diverging from the original code and laying ground work to go their own ways. Opnsense seems to be heavily rebuilding the framework and stabilizing their new code and changes. Pfsense CE and Plus seem to share the same basis at the core at this time, but Netgate does show that the two will diverge significantly over time.

I have used both, but I still see Opnsense being the better way forward. Still 100% Open source and faster on security updates and major updates. Once I got use to some of the changes, and figured it out more, I had time to build out the network to better work around my needs, plus Opnsense 21.1 released a little sooner in January, while the latest version of PFsense 2.5 was still in testing and was aiming to be completed by end of February, so it was also a matter of could not wait, and I completed all my testing with Opnsense. Now it is in production running my home network.

Sorry though if I gave any false statement. I just did not read it completely and went with what some of the community was stating in forums and searches.
 

avtella

Very Senior Member
No sorry false was a strong and wrong word on my part, I edited it to misguided.

Either way I’m glad you aren’t having to hop back and forth between the two distros and are sticking to one where everything is finally stable, finding the one that works the best for you is important and I hope it stays that way. Do keep us updated as to any changes in your network hardware or issues you encounter, your posts of your experiences so far between the two have been pretty honest and insightful.
 

Tech9

Part of the Furniture
Hey guys, do you run all this as home setup or as home lab to do fun things on it? I see you pack a lot of router power and then use a home router as AP wasting the router part of it. What is this power hungry equipment giving you more than the router itself can give in a home network? I run Netgate firewall at home, a switch and two Ruckus APs. I don't have AX support, but I don't feel the need to upgrade the APs either. My home is fairly large and my whole networking gear is under 100W including the NAS.
 

avtella

Very Senior Member
I got my SuperMicro unit to run VMs in ESXi not just pfSense alone. Right now however it's running only pfSense bare metal and yes it's way overkill for my use, and power wise around 30-32 Watts for the firewall, ~30 Watts for my switch and not sure about the two routers I haven't put my Kill-A-Watt with them yet but probably less than a 100W for my full network as well. My consumer routers that are being used as APs were given to me as part of beta tests, so I had no need to buy dedicated APs when I could just repurpose them. Actually the RAX120 was the first router I bought in like 8 years and that was before I made the move to pfSense which I had being eyeing for a few years. I finally made the move after watching some of the Lawrence Systems videos on YouTube which made pfSense a bit more approachable to a novice user like me.
 
Last edited:

Maverick009

Regular Contributor
No sorry false was a strong and wrong word on my part, I edited it to misguided.

Either way I’m glad you aren’t having to hop back and forth between the two distros and are sticking to one where everything is finally stable, finding the one that works the best for you is important and I hope it stays that way. Do keep us updated as to any changes in your network hardware or issues you encounter, your posts of your experiences so far between the two have been pretty honest and insightful.
Will do, but going to stick to Opensense for the long run it looks like. Finally becoming a semi pro in its settings, and the latest distro 21.1 fixes a lot of my previous stability and driver issues. I also understand they were in the midst of overhauling some key parts. They already released their 3rd point release bringing it to 21.1.3 and with it keep modularly updating and fixing any last minute bugs. I will keep you updated about changes including to hardware.
 

bsdsource

Regular Contributor
Hey guys, do you run all this as home setup or as home lab to do fun things on it? I see you pack a lot of router power and then use a home router as AP wasting the router part of it. What is this power hungry equipment giving you more than the router itself can give in a home network? I run Netgate firewall at home, a switch and two Ruckus APs. I don't have AX support, but I don't feel the need to upgrade the APs either. My home is fairly large and my whole networking gear is under 100W including the NAS.
pfSense is based on FreeBSD and OPNsense is based on HardendedBSD. You don't really need that much horse power to run a FreeBSD based firewall. The extra horse power can be benifical for Suricata, Sensei, Squid etc. and all the bells and whistles you want. My preference is OPNsense over pfSense and OpenBSD over OPNsense.
 

Tech9

Part of the Furniture
Thank you guys. Reusing existing equipment is good as well as learning new technology. I never tried OPNSense but I work with pfSense appliances. You know in encrypted traffic world IDS is not very effective unless you run your own proxy for the entire network with corresponding complications. Good to see enterprise solutions coming to home networks. Good job!
 

Maverick009

Regular Contributor
Hey guys, do you run all this as home setup or as home lab to do fun things on it? I see you pack a lot of router power and then use a home router as AP wasting the router part of it. What is this power hungry equipment giving you more than the router itself can give in a home network? I run Netgate firewall at home, a switch and two Ruckus APs. I don't have AX support, but I don't feel the need to upgrade the APs either. My home is fairly large and my whole networking gear is under 100W including the NAS.


I am using all this in my main home network. As I have mentioned the GT-AX11000 was purchased in part for performance and even when using it in AP mode, and with the 386 firmware, I am able to use and get most of the benefits of device, including, Link Aggregation, Tri-band wireless, Aimesh 2.0, and a few other benefits with only major feature shifted to the more powerful hardware being the firewall and DHCP router capabilities.

The Opnsense Firewall Router, is the main backbone and entry point to my network. I am slowly moving a few devices onto different subnets to separate work, VOIP, Servers, and rest of network. The segmentation in the long term helps also with experimenting and testing without bringing the network down for everyone.

Opnsense and Pfsense started just as a hobby initially to play with, then I began investing in network cards and reading more up on. Eventually I started making it part of my network and now have chosen to stick with Opnsense after some back and forth.

For me it was all about the 5 pillars and having control at every point as I have home automation slowly being added to the network, plus a Windows Server 2019 system used for NAS, Gaming server, and Plex, a Ubuntu Server Asterisk VOIP, along with various other components.

The power and performance of the network, ensures that there is no one single bottleneck even at the wireless forefront and it all is working seemless.
 

ddaenen1

Senior Member
@Maverick009 - have you considered adding another SSD and run in RAID1? An SSD is quite reliable but still. I have 2 running in ZRAID1 on my pfSense install. Better safe than sorry, especially if this is your backbone.
 

Maverick009

Regular Contributor
@Maverick009 - have you considered adding another SSD and run in RAID1? An SSD is quite reliable but still. I have 2 running in ZRAID1 on my pfSense install. Better safe than sorry, especially if this is your backbone.

I thought about it, but because there is no mechanical hard drives, and it is an SSD, I kept it to a singular drive at the moment. I however have backed up the settings, so if something did ever go wrong including a configuration, I have a restore point.

When I upgrade the hardware with a new motherboard, CPU, and Ram, I may evaluate that as an option again.

I also have been building out my Windows Server 2019 system and once I get the new drives plan on having 1-2 Raid arrays as that will be the main Data and Gaming Server. I could even add Routing server roles to it so it can act as a failsafe backup. Right now my main focus was choosing a Distro to use around the 5 pillars I mentioned and build out from there.
 

avtella

Very Senior Member
Forgot to add, it's good you got Multi-Gig... Comcast is now converting the gig plan to 1.2 Gbps so with over provisioning people are getting around 1.4 Gbps, On a standard gig ethernet connection you'd be losing almost 50% of your provisioning. So far they have upgraded East and Central regions, I'm in Comcast West but hopefully we get that change soon as well.
 
Last edited:

Maverick009

Regular Contributor
Forgot to add, it's good you got Multi-Gig... Comcast is now converting the gig plan to 1.2 Gbps so with over provisioning people are getting around 1.4 Gbps, On a standard gig ethernet connection you'd be losing almost 50% of your provisioning. So far they have upgraded West and Central regions, I'm in Comcast West but hopefully we get that change soon as well.

Actually the modem needs to support Link Aggregation (LAGG) using 2 1Gbps ports to achieve that overhead or be a newer modem that has a single 2.5Gps port to achieve faster speeds. If only using a single 1Gbps connection you will then still be limited physically by the hardware 1gbps port.
 

avtella

Very Senior Member
Yeah I wrongly assumed you had one.
 

Maverick009

Regular Contributor
Yeah I wrongly assumed you had one.

I do have a Netgear CM1200 which supports LAGG but have not enabled it yet with the latest Opnsense Distro, as I wanted to make sure everything was working correctly and stable before hand, especially knowing sometimes Netgear and Pfsense/Opnsense can be a pain and cause loss packets if not setup and synced properly.

I was debating on LAGG with the Netgear or upgrading modem to the newer model once it launches that has a single 2.5Gbps port. I will keep you posted if I go LAGG in the short term.
 

Maverick009

Regular Contributor
Small Mini update- Setup Link Aggregation (LAGG) on the cable modem and OpnSense Firewall Router. It took a moment for them to sink including two restarts of the cable modem and Opnsense before they started to work together without and bad packets through the gateway. All is good now and achieving speeds between 990Mbps-1.145Gbps wired for internet. I did also see that the new Netgear Nighthawk CM2000 and CM2050V modems are now available and I may think about picking one of them up, as they have a dedicated 2.5Gbps port on them, making the connection more seamless and allow slightly faster speeds. Definitely an option to think about.
 

Hikari

Occasional Visitor
Hey guys, do you run all this as home setup or as home lab to do fun things on it? I see you pack a lot of router power and then use a home router as AP wasting the router part of it. What is this power hungry equipment giving you more than the router itself can give in a home network? I run Netgate firewall at home, a switch and two Ruckus APs. I don't have AX support, but I don't feel the need to upgrade the APs either. My home is fairly large and my whole networking gear is under 100W including the NAS.
Hello. Talking about my case, I still have a OpenWRT router, and a Tp-Link AP where I also installed OpenWRT and run it as bridge.

I use a custom router because I want better freedom to configure my network, specially IPv6. I set on it all my devices' domain name, IPv4 addr and IPv6 suffix, all manged by dnsmasq. I also have 2 ISP (long and sad story...) so I need load balancing on them.

I needed another device as WiFi AP because my router doesn't have WiFi, and WiFi wasn't a big requirement on choosing the hardware, precisely because I could have it on another device.
 

Hikari

Occasional Visitor
Reposting from older thread.

Hello friends. I currently have a OpenWRT router and have been considering building a PC and looking for a gateway OS for it. I was looking for differences on pfSense and OPNsense and got here.

Tnx a lot for the thread comparing both. My main need is support for dual-wan multi-homing, with load balancing and failover. Most OS have that, but I also need NPTv6 with support for dynamic prefix, because both my ISP insist on providing a single /64 prefix and they claim that only state-owned ISP are forced to follow open standards. Yes, they claim they have only to offer connectivity to their intranet, and if we're unable to reach Internet it's our problem.

As of now, OpenWRT provides multi-homing with its mwan, but it only works for IPv4. For IPv6, all devices receive addresses on both prefixes and use the routing they desire. For all other VLAN, Internet is unreachable in IPv6. It has no support for NPTv6.

pfSense and I believe opnsense support NPTv6, but only with static prefix. Every time a ISP changes the prefix, I'd need to notice it and update the setting. opnsense has a task for adding support to dynamic prefix, but it's been years that nobody work on it. It seems that most devs have enough ISP competition and just hire one that provides static /56 prefix, and most users just disable IPv6.

I believe NPTv6 to be the simples solution for me, because it'd allow to provide a single prefix for all devices and keep the load balancing + failover being managed solely by the router. When my router goes down I lose Internet access, which incluces some cloud services I use, so I need to stop whatever I'm doing and fix it, so it's not an option to have a working LAN while Internet is down.

Regarding pfSense+, I also feel sad about it, but I'm not surprised. Ever since I learned about pfSense, it felt odd that Netgate gladly provides their OS for free and profit from selling appliances. On Brasil there are only 2 companies that sell them, but 1 doesn't have them for sale and the other only import on demand.
One of my requirements for the new router is to be properly able to backup and restore the storage partition, so an appliance isn't good for me.

I don't mind with them having a paid edition. RedHat and other Lix distros did that years ago and they still have their community edition rolling. If they'd require a subscription and keep it on low price, I'm wishing to pay for it. The money would keep the business sustainable and assure new features be implemented. But then, they still don't support dual wan + dynamic prefix + NPTv6. I'm not confortable to pay a subscription on a service that doesn't have the main feature I most need.

My issue is they providing it as closed source. I agree that their objective is avoiding forks, be it of the full OS or of features they develop. I'd be glad if the subscription would incentive them into developing the feature I need, but I fear they keeping it closed and opnsense and other OS be unable to use their code and I be locked on them.

On the other hand, what we've seen is that the community edition of solutions that followed that path had lost popularity. Few ppl use Fedora today. Even OpenOffice lost support compared to LibreOffice, just because their license is "less open".

In any case, I believe it's a fair move, at least for a trying. If they fail to succeed, at least I hope they move back and open the source of any feature they develop, so that at least it can be forked.

Lastly, they said that as of june pfSense+ will be available for "3rd party" hardware. Let's see how it goes, and how hard it will be to move between pfSense+, pfSense CE and opnsense, keeping existing settings.
 

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top