What's new

pfSense (or other dedicated router) questions

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

my mind immediately stumbles when I read "IP addresses change" - arent you assigning _all_ IP addresses in your network? how can they change if you map them to MAC addresses?
First, I only do static or DHCP reservations for servers of some kind. Basically, only if I have a need to know the IP address won't change.

Even then, however, those machines still have changing IPv6 addresses. If you use DHCPv6 to assign a given machine a specific IPv6, the machine might ignore DHCPv6 (such as Android does.)

In other cases, such as Windows or Mac machines, the DHCPv6 address is assigned, and then windows goes ahead and generates random temporary IPv6 addresses of it's own in the same IPv6 network in addition to the assigned address (and will use the temp address for all outgoing traffic.) Those temp addresses can (and do) change from time to time. There's a RFC for this.. I think it's called "privacy extensions" or something, but I'm not sure at the moment.

Once you realize that you can't know the IPv6 address of any given machine ahead of time (unless make ARP completely static), it becomes much more challenging to block traffic between two devices on the same L2 network if constrained to only L3 filtering.

I've only come up with two possibilities, and both require using different L2 networks. First, keep things on separate vlan's, but route selected traffic across. This works iff you aren't reliant on broadcast traffic (of if the broadcasts support IGMP.)

The second is to use pfsense bridging: put the different devices on different vlan's, but bridge the vlan's to a single L2 network. Doing this allows you to use pfsense firewall rules to selectively block some of the bridging. HOWEVER, because pfSense filtering only works at L3, I'd STILL have to disable IPv6 for at least one of the vlan's in order have effective filtering.
 
Most homes he can do in two hours at most... and CAT5e/CAT6a is cheaper by the spool, as is RG6 (upgraded coax).

Most jobs end up being under $200 - considering time/material, and one's on time/value - find an installer - I found him on the local Craigslist actually under the Jobs/Skilled Trade and Craft section... wish I would have found him before I did my own cable pulls here at the house - he would have done the same job in a quarter of the time...
I guess I picked the wrong person for an estimate then.

I wanted two pulls:

1. 10 cables from basement to attic, unterminated, unrouted - just leave 30ft of lose cable in the attic for all of them.
2. 4 cables from my home office to the basement. Again, just the pulls with no ends, etc.

I was given an estimate of 1250 for ~4 hrs of work.

In all fairness, I wanted cat6, so figure $250 for a spool of cat6. (I could get it cheaper, but I'm allowing for markup.) There are no other material costs that I'm aware of (as all the cables would be left sticking out of walls/joists, and I'd be responsible for patching any drywall.) So, $1000 for 4 hours. Assuming two people, that's $125/hr for each person for pulling wire.

I can't afford that.
 
I guess I picked the wrong person for an estimate then.

I wanted two pulls:

1. 10 cables from basement to attic, unterminated, unrouted - just leave 30ft of lose cable in the attic for all of them.
2. 4 cables from my home office to the basement. Again, just the pulls with no ends, etc.

I was given an estimate of 1250 for ~4 hrs of work.

In all fairness, I wanted cat6, so figure $250 for a spool of cat6. (I could get it cheaper, but I'm allowing for markup.) There are no other material costs that I'm aware of (as all the cables would be left sticking out of walls/joists, and I'd be responsible for patching any drywall.) So, $1000 for 4 hours. Assuming two people, that's $125/hr for each person for pulling wire.

I can't afford that.

That's actually not that bad. I paid $1,000 in metro Atlanta for 3 unterminated pulls to attic, 2 to my office, 1 to master bedroom, 1 to playroom, 1 to living room (2nd floor), and 1 to kitchen area where we keep a computer. It was all CAT6.
 
I was given an estimate of 1250 for ~4 hrs of work

Wow! That's extremely high (both scenarios actually) - must be "connectivity" in the community for me then - and they're just doing the pulls? No terms or testing?

@garyd9 - he's charging time and material - get another quote, and one thing you might ask - charge by the Drop and with/without material (e.g. you provide and/or they provide) - guys that charge by the drop do tend to be more efficient..
 
First, I only do static or DHCP reservations for servers of some kind. Basically, only if I have a need to know the IP address won't change.

Even then, however, those machines still have changing IPv6 addresses. If you use DHCPv6 to assign a given machine a specific IPv6, the machine might ignore DHCPv6 (such as Android does.)

In other cases, such as Windows or Mac machines, the DHCPv6 address is assigned, and then windows goes ahead and generates random temporary IPv6 addresses of it's own in the same IPv6 network in addition to the assigned address (and will use the temp address for all outgoing traffic.) Those temp addresses can (and do) change from time to time. There's a RFC for this.. I think it's called "privacy extensions" or something, but I'm not sure at the moment.

For my home use - I keep my "servers" on link-local addressing, and only expose V4 services, and then only on an as needed basis for outward facing ports.

Macs and Linux are pretty easy to set as Link Local, Windows is a bit more work, and sometimes things are slightly different depending on which version of Windows is in use - but the link below is a good reference from a windows perspective

https://msdn.microsoft.com/en-us/library/windows/desktop/ms739166(v=vs.85).aspx

I realize this isn't a total answer - and IPv6 does introduce a bit more complexity depending on needs/requirements...
 
For now, however, I'm taking the advice of @sfx2000 and just taking a break from it. I can use the time to get the "unifi" AP controller working on a raspberry pi, or trying to figure out how to pull ethernet through three stories of my house. (As with other things, I could just pay someone to pull the wire, but I'd learn nothing doing it that way, and it'd probably cost me 10x as much.)

Just finished that part. Using this link works perfectly for me:
http://www.lowefamily.com.au/2016/06/02/installing-ubiquiti-unifi-controller-5-on-raspberry-pi/
Follow the instructions to include Oracle JDK 8, as it runs much better on the Pi than openjdk.
 
Last edited:
On the ethernet pulls... I decided that I wasn't going to pay that much (or anywhere near that much) and did a "downsized" version of the same job myself. (Basically, I'll keep all the "server" stuff in my home office instead of moving it to the basement.) So, I spent Saturday pulling ethernet... myself. (Yes, I AM sore right now.) It wasn't a "professional" job, as I took some shortcuts. For example, instead of going through the space between floor/ceiling to move horizontally, I just ran the wire along the ceiling of a room and concealed with racetrack.

So, now I have a couple spare drops, and two unifi AP's in my house (and no area of my house has < -70 dbm on 5Ghz.)

As for the raspberry pi, those instructions on that exact same website were what I followed. Originally, I stuck everything on a raspberry pi 2 and used openJDK, but I found the WebUI to be incredibly slow. So, replaced it with a Pi3 and things sped up considerably. However, I couldn't use the android app to access the connector, so switched to Sun/Oracle JDK and that fixed THAT issue. ;) (And posted on the unifi forums the solution to that apparently common problem.)

In the meantime, I think I've decided to simplify my entire network "plan." As well, I'm going to make use of pfSense "bridging" to allow me to bridge my primary and guest vlans, but still filter between them. Despite what one troll on the pfsense forum says, pfSense actually suggests using a bridge for "Filtering between portions of a single subnet." (oh, and that particular troll can't seem to get past "you're an idiot" type posts into suggesting alternatives, or even saying why using the bridge is supposedly a bad idea. Such a "hero...")

So, I'll just go with:
  • a "management" vlan (vlan1)
  • an isolation vlan (for devices that only EVER need internet access and nothing else)
  • a guest vlan (for guest wifi)
  • a normal vlan (for everything else.)

I'll bridge guest and normal and use bridge filtering to completely block IPv6 from guest, and block guest from accessing any machine in normal with an IPv4 indicating a controlled machine (NAS, server, etc.)
 
Just finished that part. Using this link works perfectly for me:
http://www.lowefamily.com.au/2016/06/02/installing-ubiquiti-unifi-controller-5-on-raspberry-pi/
Follow the instructions to include Oracle JDK 8, as it runs much better on the Pi than openjdk.

Took me a minute - as Java 7/8 for ARM is armel, not armhf - then I realized he's not running the same distro on the Pi as I am...

He's running Raspian Jessie - I'm running Ubuntu Mate 16.04LTS for Pi2/Pi3 - perhaps the OpenJDK performance there might be a bit better...

(Java7/8 has a version for vfp, it's still armel, but supports ARMv7's that have floating point units (not all do) - VFP performs nicely, but the EABI needs to pass all floating point calls thru the integer registers first, so it's a bit of overhead.)
 
He's running Raspian Jessie - I'm running Ubuntu Mate 16.04LTS for Pi2/Pi3 - perhaps the OpenJDK performance there might be a bit better...
So, you're running unifi AP's as well?

Does the ubuntu pi distro support arm64? I'm curious if performance would be any better with all those extra registers available. (Sure, memory usage would go up of course.)
 
So, you're running unifi AP's as well?

Does the ubuntu pi distro support arm64? I'm curious if performance would be any better with all those extra registers available. (Sure, memory usage would go up of course.)

No, not running unifi's at present, I've got another vendor that for the most part meets my needs at the moment, and upgrading AP's isn't cost effective...

Going back to the Pi's - Oracle's JDK does support the FPU, but only thru the VFP instructions.

64bit - might be a long time coming with the Pi3's - even though the Cortex-A53's are ARM8 and are 64bit capable, the VC4 GPU is 32-bit, and it boots the A53's... and that's a bit of a problem... consider the Pi CPU's (all of them) as GPU's that are coupled with CPU's, which is inverse of most archs...

But the A53's do benefit from internal improvements (basic architecture), so running 32-bit ARMv7 (and legacy code) do benefit there on a DMIP's basis. Clock speed is part of the improvement (900MHz to 1200MHz) and a bit faster memory bus...

FWIW - the armhf builds also run better with the A7's in the Pi2 - and since I have just Pi2/Pi3 in the LAN, made sense to go with UbuntuMATE rather than Raspian or even direct to Debian Jessie...

A long pole here with Pi's is that the Pi Foundation is trying very hard to keep SW compatibility across all the Pi's - and the PiZero isn't ARMv7, as it's really a shrunk down Model A... all Pi's can run the code there...
 
No, not running unifi's at present, I've got another vendor that for the most part meets my needs at the moment, and upgrading AP's isn't cost effective...

I'm running Airport's as my AP's - they support it well enough, centrally managed via desktop (Mac/Win) and iOS via an App - but more importantly - they do support VLAN tagging with the "Guest" SSID over WiFi - which is unusual for AP's under $200USD... VLAN1003 is the hint there, as that is the reserved VLAN for the Guest network, even in AP mode vs. Router mode... (AP mode, Apple refers to as Bridged mode).
 
Took me a minute - as Java 7/8 for ARM is armel, not armhf - then I realized he's not running the same distro on the Pi as I am...

He's running Raspian Jessie - I'm running Ubuntu Mate 16.04LTS for Pi2/Pi3 - perhaps the OpenJDK performance there might be a bit better...

(Java7/8 has a version for vfp, it's still armel, but supports ARMv7's that have floating point units (not all do) - VFP performs nicely, but the EABI needs to pass all floating point calls thru the integer registers first, so it's a bit of overhead.)

Glad someone is worrying about architectural purity, because I'm surely not! ;-) Raspbian Jessie is working "good enough" for me. I guess that's why I'm an engineer, not a scientist :)
 
Wow.. this thread certainly got long. ;)

Pardon me while I take this into a more philosophical direction (and even ramble)...

Thinking about this more and more, I think one of the problems I'm having is that people use the generic terms "L2" and "L3", but those terms are meaningless in a vacuum. L3 can't exist without a defined L2. In addition, neither of the terms have any meaning without a defined L1.

"L3" means the network layer. Most of the L3 firewalls/routers discussed in this thread would be better called "IPv4 routers" because they require IPv4 network addressing in order to do any routing. A few also do IPv6 routing and dabble with IPSec. In most of these cases, they are also L2 devices, because they are intelligent enough to route L3 packets to other L2 devices.

My mental block is that I don't think of my PC as "192.168.0.1." It only assumes that identity when it's using IPv4 _and_ that happens to be the address DHCP assigned. I could very easily program my DHCP server to give a new IPv4 address to each machine every few minutes! As well, I can just turn off IPv4 and be completely dependent on IPv6!

I think the IPv6 issue is more concerning than the random IPv4 issue. IPv4 can be somewhat controlled and contained with DHCP. IPv6 can't be.

So, ignoring IPv4 addresses that might change and ONLY focusing on IPv6... and assuming that the L3 firewall/router has at least the pfSense level of ipv6 support... what good is a L3 router/firewall that depends on IPv6 addressing when IPv6 addressing can't be predicted? Privacy extensions are an accepted part of the IPv6 standard, which means that IPv6 addresses are DESIGNED to not be predictable!

In fact, I sampled 20 devices on my network to see what happened with DHCPv6. (The only two devices I have that don't support IPv6 weren't included.) Of those 20 devices, ZERO used the DHCPv6 address to access the internet. In some cases (android devices) the DHCPv6 server was ignored. In the rest, the device used it's own SLAAC address or a privacy extension address for outside access. There's no way to route or firewall single machines when I can't predict which IP address they'll use!

Oh, in one case (a windows machine) I even assigned it a static ipv6. It still generated a temp IPv6 for outgoing access.

In all cases where a DHCPv6 assigned or static IPv6 was accepted by the client, I could use that address for traffic going TO that client... it just wasn't usable for traffic coming FROM that client. At least this allows port forwarding to still work.

The solutions...

In the short term, with tools like pfsense, this type of thing can only be handled with generic network/subnet addressing. That means routing and firewalling have to occur at the network level, and not at the host level. While my tests show that nothing uses a DHCPv6 assigned address for outgoing access, at least they always used an address in the same /64 network. For most folks, this is what they already do anyway, so it's no big deal.

It should be noted that this means that anything except for a "default route" or "default firewall rule" will require different subnets... which means having to get an IPv6 prefix larger than /64 from your ISP. For some, that's doable. For others, it's not. It's also important that even if your ISP allows it, there aren't many router/gateway products that also support it easily. pfSense does. Sophos products do NOT. Some DDWRT/Tomato variants do, but usually require going into "advanced" settings pages or even modifying scripts.

Extending that idea... if you need all traffic on the same subnet (either because of ISP /64 limitations or for legacy "broadcast" apps), at least with pfSense, a person could use L2 vlan's bridged together to "appear" as a single subnet, and use firewalling at the interface level.

In the longer term, I think the soft/firmware for routers and firewalls is going to have to be re-designed with the problems of IPv6 planned for. We're trying to shove IPv6 concepts into IPv4 mentalities, and it just doesn't work. IPv4 assumes one IP address for each interface. IPv6 has no such limitation. It's fairly normal for routed/firewalled interfaces to be NAT'd in IPv4. That isn't the case for IPv6. IPv4 addresses can be predicted, but that also isn't the case for IPv6.

One possible way to deal with the difficulties of IPv6 routing/firewalling might be to fall back to L2. At least at the L2 layer, I have a single address for a single interface that can be predicted. It might even be considered to use ipv6 DUID's for routing purposes. (At least a DUID can be re-assigned in case a machine gets replaced.) Then again, this is trying to fit the new IPv6 problems into an older solution.

Similar to the above idea, perhaps a router/firewall should keep better track of NDP and use that data to massage firewall/routing rules. So, for example, if my router somehow knows that 2001::2:3 should have rule X applied, it watches NDP to learn the MAC address of 2001::2:3. Then, if the router/firewall notices, via NDP, that "2002::7:8" has the same MAC address, it would automagically use the 2001::2:3 rules for 2002::7:8 as well.

(You know, I actually like this last idea best of all. I could use DHCPv6 to get that initial IPv6, set firewall rules on the initial IPv6 address, and NDP could be used to associate any privacy extension addresses to the DHCPv6 assigned address. This does fall short with Android devices that reject DHCPv6 completely, however.)

Some of these ideas require more than just re-working router/firewall soft/firmware. Most of the routing and firewall "magic" actually happens in kernels, so perhaps the kernels need to be re-worked to handle IPv6 as a first class network, and not a red-headed stepchild.

By the way, if you read this far, thanks for "listening" to me ramble. :)

Take care
Gary
 
When SLAAC is used, doesn't the resulting IPv6 IP include MAC of the client device?
 
When SLAAC is used, doesn't the resulting IPv6 IP include MAC of the client device?

ONE of the IPv6 address will be EUI64 encoded. But then most systems will also generate a "privacy" to prevent protect against MAC "tracking" in a network.
See: https://tools.ietf.org/html/rfc4941

It makes centralized management of a large network a pain in the a** as these addresses are basically randomly generated by the host. There are flags to turn it off in most OS's, but the simpler solution has turned out (at least for my org) to just use DHCPv6.
 
@garyd9 Do your IPv6 devices have consistent hostnames? pfSense aliases can include hostnames.
 
@garyd9 Do your IPv6 devices have consistent hostnames? pfSense aliases can include hostnames.
Oh, they have hostnames... but that doesn't do much good. Addresses generated by privacy extensions aren't registered in DNS. The only addresses registered in DNS are static, ones created by DHCP/DHCPv6, and primary (not privacy extension) IPv6 addresses on Windows machines in AD environments.

Primary SLAAC addresses generated on a Windows machine will get registered in a AD DNS server, but those aren't the ones used for privacy extensions. Android, Mac, and iOS don't register any IPv6 addresses as far as I can see.

When SLAAC is used, doesn't the resulting IPv6 IP include MAC of the client device?
Sometimes. Not always. ;) The RFC is kind of vague using the term "interface identifier." I'm not sure if the android devices use the MAC address or some other identifier. (They might use some combination of SLAAC and privacy extensions!!)

It makes centralized management of a large network a pain in the a** as these addresses are basically randomly generated by the host. There are flags to turn it off in most OS's, but the simpler solution has turned out (at least for my org) to just use DHCPv6.
Yep. If you have an org that is ALL windows machines, AND they are all on active directory, you can use a GPO to force privacy extensions off throughout. Otherwise, have fun walking around to each and every machine, logging on as admin, and running registry scripts. Oh, and that ONLY includes windows machines. No idea what to do in linux, Mac, iOS, etc, etc, etc.

From a management perspective, using the ideas and concepts we've all grown accustomed to, ipv6 is a PITA.
 
...another case where L2 might be the answer to a L3 problem:

Several different monitoring tools understand L3 IPv4 very well. Want to know the bandwidth for John Doe's computer? Just ask the monitoring tool to query that IPv4 address, and you get instant results.

When that tool adds IPv6 support, however, things break down. The first problem is that it's very difficult to know which IPv6 address John Doe's computer might have been using today. Second, even if you do know, you have to also consider that the IPv6 address might have changed... and that reverse DNS on ipv6 is nearly useless for outgoing traffic (for the many reasons in previous posts.) Finally, some traffic might have been done with ipv6 and other traffic with ipv4. How do you know?

Even if there was only 2 or 3 ipv6 addresses, they aren't the easiest things to remember. I can remember that my desktop machine is ipv4 192.168.0.50. That doesn't change, and all I really need to remember is ".50." However, I have NFC what my desktop machine's primary global IPv6 address is. Not only does the prefix change on comcast's whim, but remembering a 64 bit suffix is.. difficult (and changes every day!)

So, the tools aren't handing the whole concept of "1 machine might have hundreds of different addresses." So, instead of that monitoring tool showing that John Doe's machine is doing a bit of youtube, a bit of streaming porn, some p2p, maybe a little FTP, etc... What I'm seeing is that 192.168.1.x is doing some porn. 2001:a:b:c::d is doing some youtube. 2001:a:y:s:t:b::s and 192.168.1.x are both doing p2p, and 2001:5:3:2::1 is messing with ftp. The tools don't seem to recognize that all those addresses belong TO THE SAME MACHINE.

The best part is that the same tool might show that the MAC address for each one of those IP addresses is.. the same. If the tool had the ability to group the data based on MAC address instead of on IP address, it would be useful for TODAY'S machines.

I came up with this one while playing with ntopng on pfsense. I knew my daughter was watching youtube, but ntopng wasn't showing it... because google's youtube prefers IPv6 and her iphone also uses privacy extensions... so tracking down the right IP address to view in ntop was a PITA. (Oddly, the "ng" versions of ntop don't seem to group on MAC address. I thought older versions did.)

I'm not saying the MAC addresses will be the Final Solution for grouping together different addresses on a single interface. What I'm saying is that some solution is needed, and MAC addresses will work for now. (As well, MAC addresses do provide a common element between IPv6 and IPv4, and both are accessible on a local network via NDP or ARP.)
 
MAC is just as bad IPv6 though. For example, some of my Android devices constantly changes their MAC address... so neither L2 nor L3 firewalling will reliably work. What happens when your son's Android device employs this technique or he learns to do it himself?

My favorite idea is to give your son his own SSID that's on a schedule.



You might look into pfSense's ability to match traffic based on the source OS in the firewall rule Advanced Options. You could maybe(?) also use DSCP tags, only passing those you allow.



Do you use a whitelist (block everything) or blacklist (allow everything) type of firewall setup? I switched to a whitelist setup as a learning exercise, allowing only certain ports, and it was much easier than expected. It offers better control.
 
MAC is just as bad IPv6 though. For example, some of my Android devices constantly changes their MAC address... so neither L2 nor L3 firewalling will reliably work.
ugh. I wasn't aware of any android devices doing that! (I know they constantly change their hostname... but I've never seen the MAC address changing.) If that's happening, then there really isn't any reliable way to specify a single device. :(


What happens when your son's Android device employs this technique or he learns to do it himself?
This isn't about my son anymore. I'm trying to think in a much broader and theoretical sense... instead of asking "how do I solve my specific issues?" (which I've mostly worked around anyway) ... I'm just discussing how the generic issues should be solved in the long term for anyone.

You might look into pfSense's ability to match traffic based on the source OS in the firewall rule Advanced Options. You could maybe(?) also use DSCP tags, only passing those you allow.
It can do that? I don't think I've seen those options. Do they require an additional package to be installed?

Do you use a whitelist (block everything) or blacklist (allow everything) type of firewall setup? I switched to a whitelist setup as a learning exercise, allowing only certain ports, and it was much easier than expected. It offers better control.
I'm really trying to use the "whitelist" mentality, but I'm finding that it's more difficult than expected. For example, I have an "isolation" vlan. For that vlan, I REALLY wanted to use rules that deny everything and just allow specific things. In my head, that went like this:
"PASS traffic to the DHCP and DNS servers on specific ports"
"PASS traffic that does NOT access another vlan"
"DENY everything else."

However, that doesn't work. The sticking point is the second rule.. there's no way to do it. If I was only using ipv4, I could do something like "PASS traffic that doesn't use private addressing", but once I introduce IPv6, that no longer works. Then I was hoping I could use firewall aliases to create an alias that included all the other vlans. THAT doesn't work either. I can't use the generic "VLAN1 net" type shortcuts in aliases. :(

(Those shortcuts seem to be the only way to include both ipv4 and ipv6 addressing when the ipv6 network addresses are dynamic based on a tracked WAN prefix delegation.)

So, my current solution is more of a blacklist instead:
PASS to DHCP and DNS on other vlan's
DENY to VLAN1 net
DENY to VLAN2 net
DENY to VLAN3 net
PASS everything else

If you have some ideas on this, PLEASE pop over to this thread on pfsense and offer the suggestion: https://forum.pfsense.org/index.php?topic=116782.0

Take care
Gary
 

Similar threads

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top