Best option for CenturyLink Gigabit fiber

[TexaClone]

So, I have been trying for the last couple of months to come up with an optimal solution for improving my CenturyLink Gigabit fiber.

I have the CL ZyXel C1100Z. Terribly slow interface, but accelerates PPPoE well and if all I was doing was browsing the web, this might be good along with a wireless AP. However, I have 7 kids, 4 Xboxes, 3 gaming PCs and a plethora of other items and Strict or Moderate NAT don't cut it. The firewall policies options on this thing are either disable or contain a host of things I don't want going on if you set it to low, medium or high. If you disable a built in policy and try to replace it with your own, it is sporadic in how it works. I am trying to disable UDP port 3074 - this gets all the Xboxes Open NAT in most cases as long as uPnP is on.

So, for a long time I had a Fortigate 60D sitting in front of the ZyXel, in the DMZ and that was OK, but more and more Xbox services need IPv6 to work along with open NAT. Not to mention, IoT devices needed IPv6 and double NAT wouldn't allow for proper operation. The IPv6 config in the Fortigate wouldn't work quite right with the 6rd setup that CL has.

So, I had an old Edgeroute Lite on the shelf and I configured that. Not bad, except I found a ton of packet loss issues. If you understand the PPPoE setup with CenturyLink, you may know the Cavium in the ERL has a bug with PPPoE offloading. Long story short - nope, not a solution.

So, I read reviews and got Eero, Orbi, Google Wifi and Velop to try out their configs. None of them handle PPPoE and VLAN well enough to make the main router. However, I did find that Orbi has amazing WiFi and works with its satellite even in bridge mode. It has now replaced my Apple Airport Extreme and the whole house has fast WiFi. Google Wifi was close - but unless it is the router, its mesh doesn't work and it doesn't do VLAN tagging.

But, to remove the NAT issue, I still need a replacement router - so I tried a pfSense box with igb nics. If you know igb nics and PPPoE, you know that there is a queue bug that drops packets like the ERL above. Processor barely registers activity, but 10% packet loss occurs. Great little device - if it worked with PPPoE well, I think it would be the winner.

So, I am kind of back at square one. C1100Z is running the fiber connection, but little issues make it less than ideal. Orbi is now the choice for WiFi, but I am not 100% locked into it.

Does anyone know of a wireless or wired router that can support gigabit speed with PPPoE, VLAN tagging on, 6rd support and doesn't have a bug that drops packets like crazy with an extensive and controllable firewall? And finally, can support Open NAT for multiple Xboxes(really just the ability to block port 3074)

 

[avtella]

I’d say go for the R7800 it’s among the best for 5Ghz band in a consumer router and decent 2.4 GHz. It supports VLAN tagging and 6rd, I would avoid 6rd regardless of router on CenturyLink, it can cause significantly higher latencies according to CTL itself.

I have the Technicolor C2100T but not on fiber, the R7800 works fine for PPPoE with transparent bridging on the C2100T. You have an extra step since you want to completely remove the ISP router.

Follow the instructions below:
https://kmwoley.com/blog/bypassing-needless-centurylink-wireless-router-on-gigabit-fiber/

 

[TexaClone]

Unfortunately, on fiber there is not a way to do transparent bridging, not like on a DSL connection. The link is active from the ONT and what comes into your home is pure ethernet. In order for the connection to work, you need VLAN tagging and PPPoE. If you keep the ISP device in play, you ALWAYS have double NAT. Thus, in order for things to all work correctly, you have to remove the ISP device. I confess I haven't used the Asus routers, but none of the other routers in consumer class I have tried to date support PPPoE and VLAN tagging with acceleration without a dropped frame bug or they don't get to full speed. I've even tried a Cisco RV340.

I think my next stab at this might be an old Intel EM driver based card with pfsense unless someone else comes up with a good idea.

 

[avtella]

Did you try the steps in the link?

 

[TexaClone]

Did you try the steps in the link?

Yes, and I have gotten every one of the routers I have tried to work without the C1100Z. The problem is, I have the full gigabit speed, just like the link mentions. And, unlike a lot of people, I can use that whole bandwidth. 7 kids plus home automation and some other devices crush most routers I've thrown at the problem.

The whole issue really lies on the PPPoE and VLAN tagging. And now that CenturyLink is about the merge with Level 3, that configuration is going to proliferate. When it's working, it's amazingly fast. The scale problem with dropped packets is so frustrating. And the CenturyLink device limitations cause all kinds of headaches.

So, I've decided to try adding an m.2 expansion card to PCIe box to my Qotom i5 fanless mini pc. The Qotom comes with 4 genuine Intel NIC ports, but with the igb driver, which has a known issue with PPPoE apparently. I found a dirt cheap HP NIC that uses an older em driver that should hopefully make pfsense work well. If it does, I'll keep the Orbi RBK50 kit and my new Frankenrouter running pfsense. If it works, I'll update here.

Was just hoping someone actually running gigabit FTTH from CenturyLink had found the magic solution.

 

[avtella]

Yeah sorry that’s all the help I can give.

 

[TexaClone]

No problem - I appreciate the feedback.

I did get a piece of advice on the PFsense forum - to install ESXi and then leverage the virtualized VMNET3 driver for the network. Hadn't thought of that before. If that works, I'll post it for future searchers.

What that will hopefully do is make it so pfsense uses the EM driver for the network connection, removing the IGB issue.

A little more straight forward on the hardware side and a lot more flexible on the software side - I hadn't thought of virtualizing my router, but if it works, its an excellent idea.

 

[TexaClone]

Just thought I would update this thread in case someone is looking in the future. I purchased a Qotom Core i5 box which is really a fanless PC with 8GB of RAM and 64GB SSD. It has 4 Intel i211 chipset ethernet ports, which on paper look to be amazing. The box was $300, so a little more than some mid range wireless routers, but should be pretty future proof.

The problem when using any router with CenturyLink Gig fiber is that you need to turn on VLAN tagging, use PPPoE on the WAN port and the ipv6 in a 6rd implementation. I have 7 kids, about 50 devices and lots of Xboxes and gaming PCs. Because PPPoE is almost universally implemented as a single threaded process, very few devices are able to accelerate it correctly and even fewer can accelerate it without packet loss.

I have tried and locked up or found lacking: Edgerouter Lite, Netgear Orbi as a router(great WiFi AP - got to stay in the house after testing, that device is awesome), Eeero, Google Wifi, Fortigate F60D, Cisco RV340, Linksys Velop(Has potential, but PPPoE, not so much), PFSense with igb hardware driver.

The ZyXel C1100Z was the only thing able to serve the house and even it has a problem with giving multiple Xboxes Open NAT because its firewall policies make blocking port 3074 but leaving other ports open difficult to impossible to get working. It also is a terrible wifi point in a 4,000 Sq. Ft. house.

So, I got some advice to setup the Qotom mentioned above as a pfSense router/firewall with ESXi and use the VMXNet3 driver. Well, that seems to finally be a good solution. In early testing, it looks like I get Open NAT on multiple Xboxes, low ping times - like 3ms - and high speed. The DSLReports speedtest shows a little bufferbloat, but not bad.

I think I finally have a solution.

One thing about gigabit internet - be careful what you wish for, because high speed with PPPoE is like oil and water.

And now that CenturyLink will own Level 3, yay! They will put PPPoE everywhere!

At least ESXi and pfsense were free to test - now to buy pfSense Gold.

 

[avtella]

I wish they used DHCP like in their former Embarq areas, I'm unfortunately in a former Qwest area.

 

[Sm3lter]

I have been struggling with this as well. I know it's a long shot but it doesn't hurt to ask...

I have the zyxel c2100z and I have trouble getting my gaming rig, Mainly oculus, to connect to others because my nat switches from open to strict (seemingly) randomly.

I also have issues with my two Xbox one s consoles maintaining an open nat as well.

Should I try a different gigabit capability router or is there an easier, less costly solution?

Everything I read above seems to apply to the older model zyxel c1100z

Truth be told I only had gigabit Ethernet for a few months and it is a bit different than the standard docsis 3.1 modem I had for years.

have a lot of time on my hands so easier is relative and I love troubleshooting... I have just been trying to solve this off and on since May or June SO even a little help or information may be able to help me out a lot!

 

[avtella]

Get a another router and put it in the ZyXel’s DMZ.

 

[TexaClone]

If you just put a router in the DMZ, you will end up with double NAT, that is the exact same situation I was in. Those Xboxes will show moderate or strict NAT and half the time you won't be able to connect to sessions.

What you have is a router, it is not performing 'modem' duties - it is a pure ethernet connection from the ONT and it is using VLAN Tag 201 and PPPoE to log into the connection.

I got tired of the kids' complaints and it is just getting worse.

So, I solved the problem completely with a pfSense box - just paid for their Gold subscription for the year. It is a total of $400 well spent.

 

[avtella]

If it’s switching from open to strict, I wonder if some of it is due to the modem, tried the C2100T or C2000A instead? Because I thought DMZ should remedy the double NAT issue at least partially. Some quirks exist in CenturyLink’s models. Granted the Technicolors have their own like HPNA turning back in upon reboot.

 

[TexaClone]

You need to block port 3074 in UPnP policy - otherwise the Xboxes will never reliably get to Open NAT. I had the C1100Z in place and you can't easily isolate port 3074 to be blocked - the CenturyLink built in security that is Low, Medium and High, has a combined policy for ports 53, 3074 and 88. If you setup an additional policy just for 3074 to block, it is not always enforced - very random. Xbox One can't properly share ports with 3074 open.

In addition, you can't use Double NAT to get an Xbox or PC gaming session to show Open NAT. If you don't block 3074, you will get open NAT on one Xbox and the others will show strict. If you block 3074, all of them can get Open NAT, but in my experience, the CenturyLink firmware blocks Open NAT randomly. If you can block 3074 in the Technicolor modem, fantastic, that may solve your problem. Leave UPnP on, but in the firewall rules block 3074 TCP and UDP.

 

[Sm3lter]

If you just put a router in the DMZ, you will end up with double NAT, that is the exact same situation I was in. Those Xboxes will show moderate or strict NAT and half the time you won't be able to connect to sessions.

What you have is a router, it is not performing 'modem' duties - it is a pure ethernet connection from the ONT and it is using VLAN Tag 201 and PPPoE to log into the connection.

I got tired of the kids' complaints and it is just getting worse.

So, I solved the problem completely with a pfSense box - just paid for their Gold subscription for the year. It is a total of $400 well spent.

Can you send me a link to the pfSense box Or Qotom Box you bought? And maybe give me a run down of How You set it up using esxi and VMXNet3 drivers? These terms are jargon to me and I am not at home to be able to do proper research on this subject.

 

[avtella]

Don’t buy the older Quad Core J1900 boxes, from what I know they won’t be supported in the next pFsense version, there are newer boxes using a Dual Core CPU but with the needed AES NI extensions for next gen pfsense, that look the same.

 

[TexaClone]

The first thing you should try is the port block - you will need to look at advanced options on the firewall settings, look at firewall rules and then see if you can block Port 3074.

The pfSense router option is much more complicated. If you don't understand what ESXi is you probably shouldn't build it that way. The total cost isn't bad, but the steps are pretty complicated. The Qotom link is here, though: https://www.amazon.com/dp/B072Q872V9/?tag=snbforums-20

You can get a lot of different configurations of that system - it is really just a fanless, low power PC - 15w at full use.

A good starting point for using pfsense with CenturyLink is here: https://kdemaria.wordpress.com/2015...-2-2-for-centurylink-gigabit-seattle-edition/
Read the comments and they can give you more tips for the components to buy.

For his tips on getting ipv6 working, check here: https://kdemaria.wordpress.com/2015...ense-2-2-2-for-ipv6-on-centurylink-gig-fiber/

Finally, pfSense has a document on setting up in ESXi: https://doc.pfsense.org/index.php/PfSense_on_VMware_vSphere_/_ESXi

The free ESXi info is here: https://my.vmware.com/web/vmware/evalcenter?p=free-esxi6

If you read through all the above and don't feel comfortable, then definitely go another route. It is a bit complex, but it works really well - I am able to get the best speeds I've gotten yet with that pfsense router setup.

I use a Netgear Orbi for wireless access - RBK50.

I like having the router and wireless separate in case some new technology comes out for wireless, I won't have to mess with my internet security setup.

Hopefully that gets you started. I recommend buying one of the Netgate products directly if building a pfSense router is too daunting. That will come with support from Netgate.

 

[TexaClone]

Don’t buy the older Quad Core J1900 boxes, from what I know they won’t be supported in the next pFsense version, there are newer boxes using a Dual Core CPU but with the needed AES NI extensions for next gen pfsense, that look the same.

Yes - excellent point. Since PPPoE is implemented as a single threaded instance, quad cores don't help you much anyway.

 

[avtella]

Looks like CTL just released the ZyXEL C3000Z AC2350 modem lol, shows a CPU speed of at least 598 Mhz in a help section image, then again probably lacks most of the QoS and advanced features as usual, not to mention a fraction as powerful as your pFsense box lol.

 

[TexaClone]

After trying several of their devices, I would suspect you get the same overlay of their configuration tools - in other words, easy to configure and most people will be happy, but now that I have this pfSense unit working well, I am not switching it out. I'll take the below - and get much more control and flexibility