Cloudflare DNS problems caused by Comcast?

[tomasm]

For the last few weeks, maybe months, I've noticed YouTube videos buffering, especially when skipping ahead. Then a few days ago several websites would not load and I was getting captcha prompts and errors when trying to login to websites. After some troubleshooting and trying other devices I remembered that I configured Cloudflare as the DNS when I bought my new router last year. Changing this back to the ISP default fixed all of my problems and webpages now load more quickly.

This isn't my first time experiencing Cloudflare problems. Some years ago I configured Windows to use Cloudflare on my PC then changed it back to default after having problems. Laer, installed WARP and had to frequently disable it when pages wouldn't load. After a while I uninstalled it (also had it on my phone for a while until it caused me to go over my data plan limit but that's a different Cloudflare related problem).

Are these problems related to Comcast not working well with Cloudflare or is Cloudflare not as great as so many claim it to be across the internet?

 

[Paliv]

It's related to two things. Location, and how your ISP routes to Cloudflare servers and which youtube servers Cloudflare points you to. The ones it points you to might not be the most optimal for Comcast's network or your location. I've found any third party DNS does point to different less optimal hulu and netflix servers than Comcast DNS servers. Comcast has a huge peering network, and their DNS is optimized for routing with those peers in mind.

 

[drinkingbird]

For the last few weeks, maybe months, I've noticed YouTube videos buffering, especially when skipping ahead. Then a few days ago several websites would not load and I was getting captcha prompts and errors when trying to login to websites. After some troubleshooting and trying other devices I remembered that I configured Cloudflare as the DNS when I bought my new router last year. Changing this back to the ISP default fixed all of my problems and webpages now load more quickly.

This isn't my first time experiencing Cloudflare problems. Some years ago I configured Windows to use Cloudflare on my PC then changed it back to default after having problems. Laer, installed WARP and had to frequently disable it when pages wouldn't load. After a while I uninstalled it (also had it on my phone for a while until it caused me to go over my data plan limit but that's a different Cloudflare related problem).

Are these problems related to Comcast not working well with Cloudflare or is Cloudflare not as great as so many claim it to be across the internet?

Comcast has their own connections to major streaming providers and their DNS will give you the best servers.

 

[Paliv]

Comcast has their own connections to major streaming providers and their DNS will give you the best servers.

More succinct answer, haha.

 

[drinkingbird]

More succinct answer, haha.

I think we were typing at the same time, didn't see yours.

 

[Tech Junky]

There's a few different things to take into consideration.

1st - CC DNS isn't great to begin with. A couple years ago I had an issue and it turned out their DNS got poisoned.

Providers also use cache servers to localize content to reduce network strain. https://support.google.com/interconnect/answer/9058809?hl=en / https://news.ycombinator.com/item?id=18792738

Captcha tends to pop more when you're using a VPN due to either blacklist of the VPN IPs or geo location issues with the end site not matching your normal location.

Using a DNS other than your provider doesn't mean it's the cause of the issues you're seeing. It's just part of the puzzle when you have issues. If the DNS provider doesn't have frequent updates to servers around it's geo location then it could pick a destination that's not optimal. If the owner of the site makes a DNS update but, doesn't push it to peers then you'll get stale info as well.

I don't use provider DNS because they tend to be flakey even in 2023. Picking reputable DNS options helps in not having issues though. If you google DNS providers you'll find lists of them to use / test. If you're using something like pihole you can add several DNS options beyond the normal primary / secondary. Even in Windows you can as well. I suppose if you SSH into a dumb router though you could add more via CLI.

 

[tomasm]

Now that I think about it, I've been getting funny things happening over the last year like unexpected captcha prompts, plus errors when attempting to complete purchases on sites that use payment services like Shopify. I think I'll stick with Comcast DNS on my router for now. I'm aware of google DNS but haven't tried it or any others. My Asus router has a list to choose from and I may experiment in the future.

 

[drinkingbird]

Now that I think about it, I've been getting funny things happening over the last year like unexpected captcha prompts, plus errors when attempting to complete purchases on sites that use payment services like Shopify. I think I'll stick with Comcast DNS on my router for now. I'm aware of google DNS but haven't tried it or any others. My Asus router has a list to choose from and I may experiment in the future.

I've played a lot with it and I've found that FIOS (my ISP) DNS is not only the fastest but also gives me the best routes to things.

Comcast has had a couple severe DNS issues in the past. FIOS is using ULLINKs old DNS setup which is very good and robust and has not caused any problems. I haven't seen any major issues with Comcast in the last several years (my mom uses it heavily and neighbors do too, and I'm the first to get a call when stuff doesn't work) so they seem to have gotten it sorted for the most part.

But either way I'd recommend using ISP DNS as primary and some other DNS secondary (I'm using Level3/Lumen secondary as they perform almost as well as FIOS and do very well with keeping their localization DB/logic up to date and not serving up advertisements and redirects etc).

The asus router will use whichever responds quicker as primary. That will most commonly be your ISP's DNS, but if you find that it is using the one you want to be "backup" then you can just pick a backup that is a bit slower. For example Level3 has at least 6 servers that will perform differently based on where you're located, and most of the 3rd parties have at least a primary and a secondary spaced pretty far apart geographically.

For Boston area FIOS I've found this to be the best setup:
Primary 71.243.0.14 Primary (this is one of FIOS's local "no redirect" servers which most people don't use since they hand their ad-serving ones out via DHCP, so it is very fast)
Secondary 4.2.2.6 - Level 3 less commonly used server that for me is the fastest out of all of theirs and only slightly slower than FIOS, and for the most part returns the same IPs as FIOS, or at least ones in the same area due to round robin load balancing.

Obviously it will be different for you as you're on a different ISP and probably a different area, just giving an example of the logic.

But as you know, speed is only one factor. With comcast, localization is very important especially if you are using streaming, so using their DNS primarily is going to benefit you a lot there, and having a slow, non comcast server as secondary will ensure it only gets used if comcast's is down, but you do have a backup if they have issues.

 

[Paliv]

There's a few different things to take into consideration.

1st - CC DNS isn't great to begin with. A couple years ago I had an issue and it turned out their DNS got poisoned.

Providers also use cache servers to localize content to reduce network strain. https://support.google.com/interconnect/answer/9058809?hl=en / https://news.ycombinator.com/item?id=18792738

Captcha tends to pop more when you're using a VPN due to either blacklist of the VPN IPs or geo location issues with the end site not matching your normal location.

Using a DNS other than your provider doesn't mean it's the cause of the issues you're seeing. It's just part of the puzzle when you have issues. If the DNS provider doesn't have frequent updates to servers around it's geo location then it could pick a destination that's not optimal. If the owner of the site makes a DNS update but, doesn't push it to peers then you'll get stale info as well.

I don't use provider DNS because they tend to be flakey even in 2023. Picking reputable DNS options helps in not having issues though. If you google DNS providers you'll find lists of them to use / test. If you're using something like pihole you can add several DNS options beyond the normal primary / secondary. Even in Windows you can as well. I suppose if you SSH into a dumb router though you could add more via CLI.

Comcast used to have regular DNS issues, but I haven't seen one in the last 5 years or more. They properly implemented DNSSEC, thus stopped redirecting any DNS queries years ago. I find them no less reliable than any third party.

 

[tomasm]

Properly configuring a backup DNS that is only used when the primary is down sounds to me like it could lead to problems. Why not manually switch as needed?

 

[Tech Junky]

Comcast used to have regular DNS issues, but I haven't seen one in the last 5 years or more. They properly implemented DNSSEC, thus stopped redirecting any DNS queries years ago. I find them no less reliable than any third party.

I can't say that has been resolved in all markets since the last time it happened I permanently switched my DNS to something other than them. It was a while ago as well in terms of years. I one went step further and kicked them to the curb and switched providers though as well. I now have no data caps and pay $50/mo all in.

 

[drinkingbird]

Properly configuring a backup DNS that is only used when the primary is down sounds to me like it could lead to problems. Why not manually switch as needed?

So only configure a single DNS and after scratching your head for a while as to why sites aren't loading, go manually change it? Not sure why anyone would want to do that, or what problems having an automatic backup would cause.

 

[Paliv]

I can't say that has been resolved in all markets since the last time it happened I permanently switched my DNS to something other than them. It was a while ago as well in terms of years. I one went step further and kicked them to the curb and switched providers though as well. I now have no data caps and pay $50/mo all in.

I wish I had a suitable alternative.

 

[drinkingbird]

I can't say that has been resolved in all markets since the last time it happened I permanently switched my DNS to something other than them. It was a while ago as well in terms of years. I one went step further and kicked them to the curb and switched providers though as well. I now have no data caps and pay $50/mo all in.

300/300 Fios is $29.99 all in here So glad I got rid of Comcast a decade ago. But I do still deal with them for others and their DNS has been fine for years now. Their email servers are still problematic but I always try to get people to use outlook or gmail rather than ISP email. I will give Comcast credit though that they do let you keep your email forever if you leave, as long as you log into it once every few months. But I'd still rather have one of the major services.

 

[Tech Junky]

300/300 Fios is $29.99

Not everyone has that option. I've tried to get fiber in a major metro area and have fiber loops around me within a couple of blocks but, no one willing to extend to serve MDUs around me. I've had prequals sent to several companies and the cost to extend is too high for the extension of the loops. For my location it's either crappy DSL, CC, or FWA. If I wanted to go back to renting then moving a couple of blocks away would hit the mark for getting fiber as a 4th option.

 

[drinkingbird]

Not everyone has that option. I've tried to get fiber in a major metro area and have fiber loops around me within a couple of blocks but, no one willing to extend to serve MDUs around me. I've had prequals sent to several companies and the cost to extend is too high for the extension of the loops. For my location it's either crappy DSL, CC, or FWA. If I wanted to go back to renting then moving a couple of blocks away would hit the mark for getting fiber as a 4th option.

Sucks. Maybe you can run 802.11bb to a house down the street

Honestly I'm not sure if I'd pick CC or FWA. Would be a tough call. Well unless CC still hasn't fixed their major packet loss issues in my area after 10 years but I'm assuming they got that figured out by now.

 

[Tech Junky]

assuming

Never with them or some of the others.

It'll be interesting when they start pushing 10ge/docsis 3.1. I suspect there will be a bandwidth drought until the provision more. It all depends on the market though

 

[dave14305]

Despite hating my monthly bill from Comcast, I do have more respect for them now, at least behind the scenes. Over at the OpenWrt forums, several of us were able to engage with Comcast's VP of something or other, and within a few weeks, he managed to fix the years-old problem of Comcast inbound traffic being marked as DSCP CS1, which was bad for CAKE and WiFi WME. Everything now shows up as the much more sensible CS0.

They didn't even know they were doing it, but once they were informed, they fixed it nationwide. Amazing in this day and age.

CAKE w/ DSCPs - cake-qos-simple

Thx to moeller0 for the heads up about this thread. I work at Comcast and am working on AQM and low latency stuff (among other topics). As part of that work I've been mucking around with DSCP values and WMM to prep for dual queue networking. I noticed the post saying packets hit the WLAN marked...

forum.openwrt.org

 

[drinkingbird]

Despite hating my monthly bill from Comcast, I do have more respect for them now, at least behind the scenes. Over at the OpenWrt forums, several of us were able to engage with Comcast's VP of something or other, and within a few weeks, he managed to fix the years-old problem of Comcast inbound traffic being marked as DSCP CS1, which was bad for CAKE and WiFi WME. Everything now shows up as the much more sensible CS0.

They didn't even know they were doing it, but once they were informed, they fixed it nationwide. Amazing in this day and age.

CAKE w/ DSCPs - cake-qos-simple

Thx to moeller0 for the heads up about this thread. I work at Comcast and am working on AQM and low latency stuff (among other topics). As part of that work I've been mucking around with DSCP values and WMM to prep for dual queue networking. I noticed the post saying packets hit the WLAN marked...

forum.openwrt.org

I'm assuming you mean Tom Karinshak, you never actually contact him, it goes to a team of high up reps, but yes they are typically able to get things done when no other department can. They replaced the old "Executive Customer Care" team. However to really get things done, the "Tom" team then needs to escalate to the highest level team that is above them, that's the team that can basically bypass anything and everything to get stuff fixed. I forget the name of that secret team but they cannot be contacted directly, the SVP "tom" team must open a ticket to them.

Even though I ditched comcast a decade ago it is still the only option for my Mom so I inevitably have to deal with them every year or two for her.

They've had several QOS designs over the years ever since rolling out phone service. I suspect that one was left over from when people realized they could mark their own VOIP traffic as high priority to make it function as well as comcast's own, so they started re-marking all inbound traffic not originating from their own modem as lower priority. Probably just never got cleaned up. Should only have been an issue between comcast customers as DSCP gets stripped off when it leave's comcast's network, but given the size of the network I can see it being a problem, the likelihood of connecting to another CC customer is fairly high. Not an issue for the vast majority of normal internet users though so probably just never got any attention.

 

[dave14305]

I'm assuming you mean Tom Karinshak

You made an a-s-s out of u and me. Jason Livingood is who we worked with. He asked for some packet captures and came back a bit later saying they found where the problem was and would be deploying a fix to all their CMTS's. They finished that last week. A very pleasant experience.

Should only have been an issue between comcast customers as DSCP gets stripped off when it leave's comcast's network,

No, all or almost all of my ingress traffic was marked CS1. But not anymore.