NAT Loopback works for HTTPS but not mail ports (993/587) — ASUS RT-AC68U limitation?

[Zoolanderbot]

Hello all!! This is my first post with needing help, so here we go!

I’m self-hosting multiple websites using nginx-proxy and companion NGINX containers. I also run a mail server via docker-mailserver (DMS), with acme-companion handling Let’s Encrypt certs. Everything is containerized, running smoothly, and externally accessible.

Here’s the issue:

From the public internet, everything works, websites and email (IMAPS 993 / SMTP 587) all route correctly via my public IP and domain.

However, from inside my local network, the behavior is inconsistent:

• Websites resolve fine — NAT loopback works for ports 443/80
• But email clients can’t connect — ports 993/587 time out

After some testing, it seems like my ASUS RT-AC68U is selectively handling loopback:

• HTTP/S traffic is handled correctly
• IMAP/SMTP traffic appears to be dropped or blocked

Is this a known shortcoming of this router? Is it normal for consumer routers to only support loopback for certain ports?

Any workarounds besides running local DNS or editing /etc/hosts? I’d rather not replace the router if this is firmware-limited, but I’m open to flashing Asuswrt-Merlin if it would help.

Thanks in advance!

 

[kuchkovsky]

After some testing, it seems like my ASUS RT-AC68U is selectively handling loopback

I can't speak for this model specifically, but I tested various ports on the RT-BE88U, and there are no issues. Either the problem is due to a misconfiguration (have you tried using the public IP directly instead of a hostname to exclude DNS-related issues?), or it's a limitation of your model.

If it really looks like a loopback issue, you can install Asuswrt-Merlin and create /jffs/configs/dnsmasq.conf.add with the following content to avoid using loopback:

address=/YOUR_PUBLIC_DOMAIN/YOUR_LOCAL_IP

 

[Zoolanderbot]

Hey @kuchkovsky, thanks for the reply!

Regardless of just the public IP or the domain host, Telnet always shows the public IP when testing, so DNS looks solid. Unfortunately the test still fails either way. If I use the local IP (e.g., 192.168.x.x), telnet connects just fine. And if I switch to a mobile hotspot, it all works too — so I’m confident the mail server and DNS are configured correctly.

I dug through the router UI and couldn’t find any NAT loopback or hairpin setting. That’s why I started thinking ASUS might only preconfigure loopback for HTTP/S ports and not mail ports.

I’ll report back once I give Merlin a shot!

 

[Zoolanderbot]

I'm working on ChatGPT with this, and its response lol:

Oof — so dig correctly resolves to 192.168.2.50, but telnet still tries to connect to 76.88.xx.xxx? That suggests your Mac isn’t actually using your router’s dnsmasq as its DNS server.

What I don't understand, if it goes out to google or where ever for DNS, why isn't just coming back to my server?

 

[sfx2000]

But email clients can’t connect — ports 993/587 time out

This smells like a certificate issue, as both those ports use SSL/TLS

Check your server/client logs

 

[Zoolanderbot]

This smells like a certificate issue, as both those ports use SSL/TLS

Check your server/client logs

Externally everything resolves and works fine, but locally, connections fail because internal devices try to reach the public IP, and my router doesn’t handle that correctly for all ports.

I’m using an ASUS router, and it appears to support NAT loopback for HTTP (like port 80/443), but not for other ports like SMTP (25), IMAP (993), or submission (587). So while web access works locally via domain name, email services break when accessed the same way.

I tried switching to ASUS Merlin firmware and added a dnsmasq override to resolve the mail domain to the local IP, which worked — but only when the router was set as the DNS server. That workaround broke Apple’s Private Relay, so it’s not viable for my household, where Apple devices are regularly used.

Right now, I’m trying to figure out how to expand NAT loopback support to more ports on this hardware and ideally without disabling system-wide privacy or adding device-specific hacks. Open to firmware suggestions or clean ways to handle this without replacing the whole network.

My next thing to test is DD-WRT, and adding a custom iptables rule to manually enable full NAT loopback. Hopefully that gives me a clean solution without compromising other services.

 

[ColinTaylor]

The NAT loopback in Asus routers doesn't differentiate between ports 80/443 and other ports. Check System Log - Port Forwarding in the webUI.

You may have to disable hardware acceleration (LAN - Switch Control) and/or AiProtection. Do that when testing.

And as the previous post suggested, check the logs on the server as the traffic might be reaching it OK but not being returned.

 

[Zoolanderbot]

I tried turning off NAT hardware acceleration, so everything is now processed through the CPU. AiProtection was already off. Still no luck.

I’m using Telnet to test local connectivity from my Mac (which is still a valid):
(3.10.13) derekdreblow@MacBookPro WebSite_DreblowDesigns % telnet mail.dreblowdesigns.com 993
Trying 75.80.156.238...
^C
(3.10.13) derekdreblow@MacBookPro WebSite_DreblowDesigns % telnet mail.dreblowdesigns.com 443
Trying 75.80.156.238...
Connected to mail.dreblowdesigns.com.
Escape character is '^]'.

So port 443 works locally via domain, but IMAPS (993) fails, confirming that NAT loopback is only functional for HTTP/HTTPS on this router, not for email ports (at least to my limited understanding!!)

The end goal is to access my mail server locally using the same public domain (FQDN) that external clients use.

With these results, it seems to tell me the certs are good, NAT Loop back does work for HTTPS, and services like private reply works. That's why I can't shake off the feeling that the NAT loop back is specific to a two ports 80/443. But! I really don't know and still don't truly understand the system. I tried the logs, and I didn't see anything unique.

Given all the research, I see that the messages usually do make it to the destination but the response is usually in the local ip and not the public ip, so the ip tables drops the message since its not in the correct format. I guess the router firewall has to change it back? idk, something along those lines.

Since my next lead is ip tables, I'll add this to DD-WRT:
iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -d 192.168.1.0/24 -j MASQUERADE

 

[sfx2000]

Externally everything resolves and works fine, but locally, connections fail because internal devices try to reach the public IP, and my router doesn’t handle that correctly for all ports.

Still sounds like a cert issue - if you can connect from the WAN, then you know the servers and certs are correct.

The cert is tied to the domain, so if your internal domain is different... the SSL handshake will fail..

You may be able to fix this by adding an A record to your DNS for the internal domain

 

[Zoolanderbot]

Still sounds like a cert issue - if you can connect from the WAN, then you know the servers and certs are correct.

The cert is tied to the domain, so if your internal domain is different... the SSL handshake will fail..

You may be able to fix this by adding an A record to your DNS for the internal domain

Doesn't this only applies if the internal client is connecting using a different domain name?

But in my case, I'm using the correct public domain (mail.dreblowdesigns.com), even internally. My testing is successful via telnet mail.dreblowdesigns.com 443 but fails via telnet mail.dreblowdesigns.com 993.

 

[kuchkovsky]

Doesn't this only applies if the internal client is connecting using a different domain name?

To exclude possible certificate-related issues, try running this from any LAN device:

nc -zv YOUR_PUBLIC_IP 587
nc -zv YOUR_PUBLIC_IP 993

If these commands print "connection refused" or just hang, it means that hairpin is not working. If they succeed, it means there is some other issue, possibly with certificates.

I tried switching to ASUS Merlin firmware and added a dnsmasq override to resolve the mail domain to the local IP, which worked — but only when the router was set as the DNS server. That workaround broke Apple’s Private Relay, so it’s not viable for my household, where Apple devices are regularly used.

If using the local DNS server doesn't work for you, you can try adding these rules to your /jffs/scripts/nat-start:

iptables -t nat -A PREROUTING -d YOUR_PUBLIC_IP -i br0 \
    -p tcp --dport 587 -j DNAT --to-destination YOUR_LOCAL_IP:587

iptables -t nat -A PREROUTING -d YOUR_PUBLIC_IP -i br0 \
    -p tcp --dport 993 -j DNAT --to-destination YOUR_LOCAL_IP:993

They enable hairpin-like functionality for the specified ports using DNAT.

 

[Zoolanderbot]

To exclude possible certificate-related issues, try running this from any LAN device:

nc -zv YOUR_PUBLIC_IP 587
nc -zv YOUR_PUBLIC_IP 993

If these commands print "connection refused" or just hang, it means that hairpin is not working. If they succeed, it means there is some other issue, possibly with certificates.


If using the local DNS server doesn't work for you, you can try adding these rules to your /jffs/scripts/nat-start:

iptables -t nat -A PREROUTING -d YOUR_PUBLIC_IP -i br0 \
    -p tcp --dport 587 -j DNAT --to-destination YOUR_LOCAL_IP:587

iptables -t nat -A PREROUTING -d YOUR_PUBLIC_IP -i br0 \
    -p tcp --dport 993 -j DNAT --to-destination YOUR_LOCAL_IP:993

They enable hairpin-like functionality for the specified ports using DNAT.

Ohhh man I had the worst thing happen to me.. it works!! (no jokes) I don't know what changed!! Like an idiot I made some final changes like turning off DDNS on the router. I'm running stock firmware too, I bought two router just to tinker. One router out of box OEM and the other Merlin. The one that is working is OEM... so it seems that strongly suggests it was indeed the certs... wow!! I use ACME-Companion container for Let's Encrypt and maybe something kind of update took place.

First thing I tried was your idea of

Last login: Wed Jul  9 16:00:15 on ttys005
derekdreblow@MacBookPro ~ % nc -zv 75.80.156.238 993
Connection to 75.80.156.238 port 993 [tcp/imaps] succeeded!
derekdreblow@MacBookPro ~ % nc -zv 75.80.156.238 587
Connection to 75.80.156.238 port 587 [tcp/submission] succeeded!
derekdreblow@MacBookPro ~ %

my eyes got big so I checked Apple Mail and it worked!

Usually there is a triangle with ! inside of it, but not today.

Thank you for the help!! Even though it's up in the air, I still really appreciate your time to this and the help you tried to give.

Now I need to comb over the settings and report. Helpfully it can help out the next poor soul.