FTPS / FTP TLS WAN access broken again?

[sbsnb]

Disabling TLS and connecting using the same client and settings on mobile Hotspot, connects fine. I've described this earlier, and thus I'm sure the client/connection info is not the problem

Have you checked whether certain ports are being blocked by your hotspot provider, or your ISP?

EDIT - I'm wondering if the server is working correctly, but the connection from the server to the client is being blocked.

 

[jappish84]

Have you checked whether certain ports are being blocked by your hotspot provider, or your ISP?

EDIT - I'm wondering if the server is working correctly, but the connection from the server to the client is being blocked.

It crossed my mind aswell, but I have tried it (with the same error) from two different ISPs, and i've tried accessing the FTP server while having my hotspot connected to a VPN aswell.

I just now tried setting up a FTPES server on my Windows machine, just to try and connect from WAN, and I had no issues, although using a different port number.

Correct me if I'm wrong, but TLS (explicit mode) uses port 21, and I imagine the router configures itself correctly (well maybe not in my case)

Since I can connect to the server and the client fails at "502 Command not implemented" I'm guessing it's not a port blocket issue.

 

[ColinTaylor]

Since I can connect to the server and the client fails at "502 Command not implemented" I'm guessing it's not a port blocket issue.

Can you compare the log of a working connection to see whether you get the same 502 error for TLS. Your connection is actually failing on the SSL command so I'd like to see if the working connection is only using SSL.

 

[sbsnb]

Since I can connect to the server and the client fails at "502 Command not implemented" I'm guessing it's not a port blocket issue.

It's actually failing because of a closed connection:

10:44:38 Error:            Could not read from socket: ECONNRESET - Connection reset by peer

That and the fact that it works from the LAN leads me to believe your ISP (the one hosting your home connection) is blocking ports.

EDIT - Nevermind. I see that it's the SSL part that failed on the connection.

 

[sbsnb]

You might also try setting nvram variable enable_ftp_log=1. This should create
/var/log/vsftpd.log which might have more information.

 

[octopus]

Yes, the config requires FTPES when connecting when TLS is enabled (I don't think you can change this in the webui?)

That is setting in FTP-client. Have you configured in client PASV?

I have in my vsftpd.postconf-file

#!/bin/sh

CONFIG=$1
source /usr/sbin/helper.sh
pc_append "use_sendfile=NO" $CONFIG
pc_append "require_ssl_reuse=NO" $CONFIG
pc_append "force_local_data_ssl=YES" $CONFIG
pc_append "force_local_logins_ssl=YES" $CONFIG

and in vsftpd.conf look like this

anonymous_enable=NO
nopriv_user=root
write_enable=YES
local_enable=YES
chroot_local_user=YES
local_umask=000
dirmessage_enable=NO
xferlog_enable=NO
syslog_enable=NO
connect_from_port_20=YES
use_localtime=YES
listen=YES
pasv_enable=YES
pasv_min_port=57530
pasv_max_port=57560
tcp_wrappers=NO
max_clients=5
ftp_username=anonymous
ssl_enable=YES
rsa_cert_file=/etc/cert.pem
rsa_private_key_file=/etc/key.pem
use_sendfile=NO
require_ssl_reuse=NO
force_local_data_ssl=YES
force_local_logins_ssl=YES

 

[jappish84]

Can you compare the log of a working connection to see whether you get the same 502 error for TLS. Your connection is actually failing on the SSL command so I'd like to see if the working connection is only using SSL.

Sorry, not sure what server you are referring to, the one I set up on the Win machine? That's the one I can connect to with TLS enabled:

19:10:11 Status:          Resolving address of my.domain.name
19:10:11 Status:          Connecting to XX.XXX.XX.XX:22202...
19:10:11 Status:          Connection established, waiting for welcome message...
19:10:11 Response:     220-FileZilla Server 0.9.60 beta
19:10:11 Response:     220-written by Tim Kosse ([email protected])
19:10:11 Response:     220 Please visit https://filezilla-project.org/
19:10:11 Command:    AUTH TLS
19:10:12 Response:     234 Using authentication type TLS
19:10:12 Status:          Initializing TLS...
19:10:12 Status:          Verifying certificate...
19:10:12 Status:          TLS connection established.
19:10:12 Command:    USER jasse
19:10:12 Response:     331 Password required for jasse
19:10:12 Command:    PASS ***********
19:10:12 Response:     230 Logged on
19:10:12 Command:    PBSZ 0
19:10:12 Response:     200 PBSZ=0
19:10:12 Command:    PROT P
19:10:12 Response:     200 Protection level set to P
19:10:12 Status:          Logged in
19:10:12 Status:          Retrieving directory listing...
19:10:12 Command:    PWD
19:10:12 Response:     257 "/" is current directory.
19:10:12 Status:          Directory listing of "/" successful

@sbsnb

I tried enabling the log simply by running

nvram set enable_ftp_log=1

and restarting the server, but no log. Am I using it wrong?



@octopus
I'm not the sure if vsftpd is configured to listen for implicit FTP over TLS. This is configurable on the Filezilla Server, where you are able to set Explicit mode only if wanted.

I have tried both "Active" and "Passive" connections, both resulting in the same error.

I have no postconf-scripts enabled for vsftpd, this is my

vsftpd.conf

anonymous_enable=NO
nopriv_user=root
write_enable=YES
local_enable=YES
chroot_local_user=YES
local_umask=000
dirmessage_enable=NO
xferlog_enable=NO
syslog_enable=NO
connect_from_port_20=YES
use_localtime=YES
listen=YES
pasv_enable=YES
pasv_min_port=57530
pasv_max_port=57560
tcp_wrappers=NO
max_clients=5
ftp_username=anonymous
ftpd_banner=Welcome to ASUS RT-AC3200 FTP service.
ssl_enable=YES
rsa_cert_file=/etc/cert.pem
rsa_private_key_file=/etc/key.pem

 

[jappish84]

EDIT: (post got too big)

@ColinTaylor

Sorry, maybe you were referring to a successful connection to the routers FTP-server from the LAN side, so here's the debug-log from that connection with TLS enabled:

20:07:21 Trace:           CControlSocket::SendNextCommand()
20:07:21 Trace:           CFtpLogonOpData::Send() in state 0
20:07:21 Status:          Resolving address of domain.asuscomm.com
20:07:21 Status:          Connecting to xx.xxx.xxx.xx:21...
20:07:21 Status:          Connection established, waiting for welcome message...
20:07:21 Trace:           CFtpControlSocket::OnReceive()
20:07:21 Response:     220 Welcome to ASUS RT-AC3200 FTP service.
20:07:21 Trace:           CFtpLogonOpData::ParseResponse() in state 1
20:07:21 Trace:           CControlSocket::SendNextCommand()
20:07:21 Trace:           CFtpLogonOpData::Send() in state 2
20:07:21 Command:    AUTH TLS
20:07:21 Trace:           CFtpControlSocket::OnReceive()
20:07:21 Response:     234 Proceed with negotiation.
20:07:21 Trace:           CFtpLogonOpData::ParseResponse() in state 2
20:07:21 Status:          Initializing TLS...
20:07:21 Trace:           tls_layer_impl::client_handshake()
20:07:21 Trace:           tls_layer_impl::continue_handshake()
20:07:21 Trace:           TLS handshake: About to send CLIENT HELLO
20:07:21 Trace:           TLS handshake: Sent CLIENT HELLO
20:07:21 Trace:           tls_layer_impl::on_send()
20:07:21 Trace:           tls_layer_impl::continue_handshake()
20:07:21 Trace:           tls_layer_impl::on_read()
20:07:21 Trace:           tls_layer_impl::continue_handshake()
20:07:21 Trace:           tls_layer_impl::on_read()
20:07:21 Trace:           tls_layer_impl::continue_handshake()
20:07:21 Trace:           TLS handshake: Received HELLO RETRY REQUEST
20:07:21 Trace:           TLS handshake: Processed HELLO RETRY REQUEST
20:07:21 Trace:           TLS handshake: About to send CLIENT HELLO
20:07:21 Trace:           TLS handshake: Sent CLIENT HELLO
20:07:22 Trace:           tls_layer_impl::on_read()
20:07:22 Trace:           tls_layer_impl::continue_handshake()
20:07:22 Trace:           TLS handshake: Received SERVER HELLO
20:07:22 Trace:           TLS handshake: Processed SERVER HELLO
20:07:22 Trace:           TLS handshake: Received ENCRYPTED EXTENSIONS
20:07:22 Trace:           TLS handshake: Processed ENCRYPTED EXTENSIONS
20:07:22 Trace:           TLS handshake: Received CERTIFICATE REQUEST
20:07:22 Trace:           TLS handshake: Processed CERTIFICATE REQUEST
20:07:22 Trace:           TLS handshake: Received CERTIFICATE
20:07:22 Trace:           TLS handshake: Processed CERTIFICATE
20:07:22 Trace:           TLS handshake: Received CERTIFICATE VERIFY
20:07:22 Trace:           TLS handshake: Processed CERTIFICATE VERIFY
20:07:22 Trace:           TLS handshake: Received FINISHED
20:07:22 Trace:           TLS handshake: Processed FINISHED
20:07:22 Trace:           TLS handshake: About to send CERTIFICATE
20:07:22 Trace:           TLS handshake: Sent CERTIFICATE
20:07:22 Trace:           TLS handshake: About to send FINISHED
20:07:22 Trace:           TLS handshake: Sent FINISHED
20:07:22 Trace:           TLS Handshake successful
20:07:22 Trace:           Protocol: TLS1.3, Key exchange: ECDHE-RSA, Cipher: AES-256-GCM, MAC: AEAD
20:07:22 Trace:           tls_layer_impl::verify_certificate()
20:07:22 Status:          Verifying certificate...
20:07:22 Trace:           CFtpControlSocket::SetAsyncRequestReply
20:07:22 Status:          TLS connection established.
20:07:22 Trace:           CControlSocket::SendNextCommand()
20:07:22 Trace:           CFtpLogonOpData::Send() in state 6
20:07:22 Command:    USER jasse
20:07:22 Trace:           CFtpControlSocket::OnReceive()
20:07:22 Trace:           TLS handshake: Received NEW SESSION TICKET
20:07:22 Trace:           TLS handshake: Processed NEW SESSION TICKET
20:07:22 Trace:           gnutls_record_recv returned spurious EAGAIN
20:07:22 Trace:           TLS handshake: Received NEW SESSION TICKET
20:07:22 Trace:           TLS handshake: Processed NEW SESSION TICKET
20:07:22 Trace:           gnutls_record_recv returned spurious EAGAIN
20:07:22 Response:     331 Please specify the password.
20:07:22 Trace:           CFtpLogonOpData::ParseResponse() in state 6
20:07:22 Trace:           CControlSocket::SendNextCommand()
20:07:22 Trace:           CFtpLogonOpData::Send() in state 6
20:07:22 Command:    PASS ****************
20:07:22 Trace:           tls_layer_impl::on_read()
20:07:22 Trace:           CFtpControlSocket::OnReceive()
20:07:22 Response:     230 Login successful.
20:07:22 Trace:           CFtpLogonOpData::ParseResponse() in state 6
20:07:22 Trace:           CControlSocket::SendNextCommand()
20:07:22 Trace:           CFtpLogonOpData::Send() in state 10
20:07:22 Command:    OPTS UTF8 ON
20:07:22 Trace:           tls_layer_impl::on_read()
20:07:22 Trace:           CFtpControlSocket::OnReceive()
20:07:22 Response:     200 Always in UTF8 mode.
20:07:22 Trace:           CFtpLogonOpData::ParseResponse() in state 10
20:07:22 Trace:           CControlSocket::SendNextCommand()
20:07:22 Trace:           CFtpLogonOpData::Send() in state 11
20:07:22 Command:    PBSZ 0
20:07:22 Trace:           tls_layer_impl::on_read()
20:07:22 Trace:           CFtpControlSocket::OnReceive()
20:07:22 Response:     200 PBSZ set to 0.
20:07:22 Trace:           CFtpLogonOpData::ParseResponse() in state 11
20:07:22 Trace:           CControlSocket::SendNextCommand()
20:07:22 Trace:           CFtpLogonOpData::Send() in state 12
20:07:22 Command:    PROT P
20:07:22 Trace:           tls_layer_impl::on_read()
20:07:22 Trace:           CFtpControlSocket::OnReceive()
20:07:22 Response:     200 PROT now Private.
20:07:22 Trace:           CFtpLogonOpData::ParseResponse() in state 12
20:07:22 Status:          Logged in
20:07:22 Trace:           Measured latency of 6 ms
20:07:22 Trace:           CFtpControlSocket::ResetOperation(0)
20:07:22 Trace:           CControlSocket::ResetOperation(0)
20:07:22 Trace:           CFtpLogonOpData::Reset(0) in state 15
20:07:22 Trace:           CFileZillaEnginePrivate::ResetOperation(0)
20:07:22 Status:          Retrieving directory listing...
20:07:22 Trace:           CControlSocket::SendNextCommand()
20:07:22 Trace:           CFtpListOpData::Send() in state 0
20:07:22 Trace:           CFtpChangeDirOpData::Send() in state 0
20:07:22 Trace:           CFtpChangeDirOpData::Send() in state 1
20:07:22 Command:    PWD
20:07:22 Trace:           tls_layer_impl::on_read()
20:07:22 Trace:           CFtpControlSocket::OnReceive()
20:07:22 Response:     257 "/" is the current directory
20:07:22 Trace:           CFtpChangeDirOpData::ParseResponse() in state 1
20:07:22 Trace:           CFtpControlSocket::ResetOperation(0)
20:07:22 Trace:           CControlSocket::ResetOperation(0)
20:07:22 Trace:           CFtpChangeDirOpData::Reset(0) in state 1
20:07:22 Trace:           CFtpListOpData::SubcommandResult(0) in state 1
20:07:22 Trace:           CControlSocket::SendNextCommand()
20:07:22 Trace:           CFtpListOpData::Send() in state 2
20:07:22 Trace:           CFtpControlSocket::ResetOperation(0)
20:07:22 Trace:           CControlSocket::ResetOperation(0)
20:07:22 Trace:           CFtpListOpData::Reset(0) in state 2
20:07:22 Status:          Directory listing of "/" successful
20:07:22 Trace:           CFileZillaEnginePrivate::ResetOperation(0)

 

[ColinTaylor]

@ColinTaylor

Sorry, maybe you were referring to a successful connection to the routers FTP-server from the LAN side, so here's the debug-log from that connection with TLS enabled:

20:07:21 Status:          Resolving address of domain.asuscomm.com
20:07:21 Status:          Connecting to xx.xxx.xxx.xx:21...
20:07:21 Status:          Connection established, waiting for welcome message...
20:07:21 Response:     220 Welcome to ASUS RT-AC3200 FTP service.
20:07:21 Command:    AUTH TLS
20:07:21 Response:     234 Proceed with negotiation.
20:07:21 Status:          Initializing TLS...
:

Yes that was what I was referring to.

So now we can see that the connection is not "failing", but when accessed from the WAN the ftp server is sending back a "Command not implemented" status for TLS. So the next question is why?

 

[sbsnb]

Grasping at straws here, but are there any port mappings active ... even upnp ones?

 

[jappish84]

Maybe the vsftpd log can give some clues. How do I properly enable it?

@sbsnb

Here's everything I can find:

DNS over TLS:

2001:4860:4860::8888        dns.google      
2001:4860:4860::8844        dns.google      
8.8.8.8        dns.google

Port forwarding

server1    57121    57121    192.168.1.121    BOTH      
server 2    22202:22302        192.168.1.202    BOTH      
server 3    80    80    192.168.1.199    TCP      
server 4    8123    8123    192.168.1.234    TCP      
server 5   8883    8883    192.168.1.234    TCP

UPNP

Edit, forgot VPN

VPN server: 1194

VPN client: 1198

 

[sbsnb]

Does this guide work?

 

[sbsnb]

It may not be clear from that page that xferlog_std_format has to be disabled for log_ftp_protocol to function.

 

[RMerlin]

Tested on my RT-AC3200, working for me.

Status:    Connecting to 192.168.10.219:21...
Status:    Connection established, waiting for welcome message...
Response:    220 Welcome to ASUS RT-AC3200 FTP service.
Command:    AUTH TLS
Response:    234 Proceed with negotiation.
Status:    Initializing TLS...
Status:    Verifying certificate...
Status:    TLS connection established.
Command:    USER admin
Response:    331 Please specify the password.
Command:    PASS *********
Response:    230 Login successful.
Command:    OPTS UTF8 ON
Response:    200 Always in UTF8 mode.
Command:    PBSZ 0
Response:    200 PBSZ set to 0.
Command:    PROT P
Response:    200 PROT now Private.
Status:    Logged in
Status:    Retrieving directory listing...
Command:    PWD
Response:    257 "/" is the current directory
Status:    Directory listing of "/" successful

Are both /etc/cert.pem and /etc/cert.key present on your router?

Check the content of the certificate:

openssl x509 -in /etc/cert.pem -noout -text

 

[jappish84]

@RMerlin

Thank you for testing. Maybe I'm misunderstanding, but it looks like you are connecting from within LAN?

Status:    Connecting to 192.168.10.219:21...

I checked if the files are present; "/etc/cert.pem" is there, "/etc/cert.key" isn't present, I do however have have a "/etc/key.pem" file present. The "/etc/vsftpd.conf" file is pointing to "/etc/key.pem" and the contents of /etc/cert.pem looks good as far as I can tell. "nano /etc/key.pem" looks like a normal RSA Private Key as far as I can tell

@sbsnb

Thanks, how do I set the settings? I changed /etc/vsftpd.conf according to the link you posted, meaning I added "xferlog_std_format=NO" and "log_ftp_protocol=YES"

I then ran nvram set /etc/vsftpd.conf and restarted the FTP-server from web-ui but /etc/vsftpd.conf changed to the default value again.

This is the only thing logged:

Mon Dec 30 12:54:27 2019 [pid 19534] CONNECT: Client "redacted"
Mon Dec 30 12:54:32 2019 [pid 19538] CONNECT: Client "redacted"

 

[ColinTaylor]

Tested on my RT-AC3200, working for me.

His issue is that it works when connecting from the LAN but returns "502 Command not implemented" for AUTH TLS when connecting from the WAN.

 

[RMerlin]

Thank you for testing. Maybe I'm misunderstanding, but it looks like you are connecting from within LAN?

No, that was from WAN. My test RT-AC3200 sits within my LAN, so my laptop was accessing it from its WAN side.

I checked if the files are present; "/etc/cert.pem" is there, "/etc/cert.key" isn't present, I do however have have a "/etc/key.pem" file present. The "/etc/vsftpd.conf" file is pointing to "/etc/key.pem" and the contents of /etc/cert.pem looks good as far as I can tell. "nano /etc/key.pem" looks like a normal RSA Private Key as far as I can tell

Typo on my part, sorry.

His issue is that it works when connecting from the LAN but returns "502 Command not implemented" for AUTH TLS when connecting from the WAN.

Which could possibly have been tied to a missing or invalid certificate.

The FTP daemon has no notion of what is WAN or LAN, all it does is listen to any incoming connections. If behaviour changes depending on the origin, then it means something else in his environment (could be local or ISP-side) is interfering with it. I can't see how the FTP daemon could suddenly "lose" TLS support depending on from where you are connecting. I'd suspect a transparent proxy or something like that interfering with his setup rather than a firmware issue. Beside, I haven't touched the FTP code in months.

Could also be a Filezilla issue. It might complain about the certificate based with the server address at connection time (since SAN might not match, and/or it's self-signed), and you told it to memorize it at one point (since certificates are associated with the server address).

 

[jappish84]

The FTP daemon has no notion of what is WAN or LAN, all it does is listen to any incoming connections. If behaviour changes depending on the origin, then it means something else in his environment (could be local or ISP-side) is interfering with it.

If the FTP daemon has no notion whats coming from where, then what has changed regarding TLS? Remember, I can still connect to the FTP-server from WAN if I disable TLS on the FTP Server Config page.

Is there a way to test this further? I've already tried setting up another FTP over TLS-server behind the router on my Windows machine, and that one, I could connect to. I can check with my ISP if whey are blocking any ports, but what ports? Explicit FTP over TLS is on port 21 is what I could find.

Question: Can I disable the FTP server on the router, and reconfigure the FTP-server on my Windows machine to use port 21? Would that test if ISP is blocking that port when on TLS?

Could also be a Filezilla issue. It might complain about the certificate based with the server address at connection time (since SAN might not match, and/or it's self-signed), and you told it to memorize it at one point (since certificates are associated with the server address).

The router has new self-signed certificates since I did reset the router to factory defaults (The old certificate actually expired a few days before I noticed the issue, and let's encrypt couldn't update it for some reason until I disabled, and reanabled let's encrypt). I have since then been propted to accept/decline the certificate when connecting (I think from LAN, not WAN). I have also tried using Cyberduck FTP client (never used before), and Kodis FTP client.

 

[RMerlin]

f the FTP daemon has no notion whats coming from where, then what has changed regarding TLS?

Nothing. 384.13_2 only contained a few minor bug fixes, like switching to the ACMEv2 capable client for Let's Encrypt.

Question: Can I disable the FTP server on the router, and reconfigure the FTP-server on my Windows machine to use port 21? Would that test if ISP is blocking that port when on TLS?

Yes, just make sure to forward port 21 to your PC.

 

[RMerlin]

The odd part is your server seems to report it does not support TLS. Try querying the feature list using telnet, with the "feat" command:

$ telnet rtac3200 21
Trying 192.168.10.219...
Connected to rtac3200.lostrealm.lan.
Escape character is '^]'.
220 Welcome to ASUS RT-AC3200 FTP service.
feat
211-Features:
 AUTH TLS
 EPRT
 EPSV
 MDTM
 PASV
 ICNV
 PBSZ
 PROT
 REST STREAM
 SIZE
 TVFS
 UTF8
211 End
quit
221 Goodbye.