AC86U: unable to VPN, can't access GUI, how can I fix this via command line?

  • ATTENTION! As of November 1, 2020, you are not able to reply to threads 6 months after the thread is opened if there are more than 500 posts in the thread.
    Threads will not be locked, so posts may still be edited by their authors.
    Just start a new thread on the topic to post if you get an error message when trying to reply to a thread.

Ohia

Occasional Visitor
For some reason I've been unable to VPN into my AC86U since a week or two ago... it sits in my second home, so I have no physical access to it right now. Attempts to connect just time out. At first I thought my internet must be down, but I was able to connect to my Synology NAS using QuickConnect... and from there (using the web interface) I was able to to start up a Debian docker which finally let me ssh into the router itself. Since I don't have access to the GUI, how much troubleshooting can I do via the command line? I tried rebooting already, as well as
Code:
service stop_vpnserver1
service start_vpnserver1
but to no avail. In fact, the syslog shows:
Code:
Aug 31 03:22:37 vpnserver1[14886]: Diffie-Hellman initialized with 2048 bit key
Aug 31 03:22:37 vpnserver1[14886]: TUN/TAP device tun21 opened                
Aug 31 03:22:37 vpnserver1[14886]: TUN/TAP TX queue length set to 100          
Aug 31 03:22:37 vpnserver1[14886]: /sbin/ifconfig tun21 10.8.0.1 pointopoint 10.8.0.2 mtu 1500                                                                
Aug 31 03:22:38 vpnserver1[14886]: /bin/sh /jffs/etc/profile tun21 1500 1622 10.8.0.1 10.8.0.2 init                                                            
Aug 31 03:22:38 vpnserver1[14886]: WARNING: Failed running command (--up/--down): external program exited with error status: 2                                
Aug 31 03:22:38 vpnserver1[14886]: Exiting due to fatal error
Any ideas what could be causing this fatal error? I didn't perform any config changes so unsure why VPN access has stopped working. Thanks for any pointers, or maybe suggestions on what other parameters to check....
 

ColinTaylor

Part of the Furniture
Code:
Aug 31 03:22:38 vpnserver1[14886]: /bin/sh /jffs/etc/profile tun21 1500 1622 10.8.0.1 10.8.0.2 init                                                           
Aug 31 03:22:38 vpnserver1[14886]: WARNING: Failed running command (--up/--down): external program exited with error status: 2                               
Aug 31 03:22:38 vpnserver1[14886]: Exiting due to fatal error
What firmware version are you using?

Looks like your router might be infected with malware. See this thread and check the contents of /jffs/etc/profile.

Have you ever enabled remote access to the router's web interface?
 

RMerlin

Asuswrt-Merlin dev
Aug 31 03:22:38 vpnserver1[14886]: /bin/sh /jffs/etc/profile tun21 1500 1622 10.8.0.1 10.8.0.2 init
This looks like malware to me.
 

Ohia

Occasional Visitor
What firmware version are you using?
I am not sure. Pretty up to date. Is there a way I can check this from the command line?

Looks like your router might be infected with malware. See this thread and check the contents of /jffs/etc/profile.

/jffs/etc/profile is empty - actually so is all of /jffs/etc, but I'll have another read through the thread that you linked.

Have you ever enabled remote access to the router's web interface?

Is that the "Enable Web Access From WAN" setting? That was definitely off...

Anyway, thanks for your responses!
 
Last edited:

ColinTaylor

Part of the Furniture
I am not sure. Pretty up to date. Is there a way I can check this from the command line?
When you first log on is says something like this:
Rich (BB code):
ASUSWRT-Merlin RT-AX86U 386.3_2 Fri Aug 6 21:47:27 UTC 2021

Is that the "Enable Web Access From WAN" setting? That was definitely off...
What about AiDisk or AiCloud?
 

RMerlin

Asuswrt-Merlin dev
For us mere mortals, can you say what exactly about that line makes you think malware?
Thanks RMerlin.
OpenVPN shouldn`t be executing that file, and I know at least one malware which installs itself into both openvpn-event and /etc/profile .
 

Ohia

Occasional Visitor
When you first log on is says something like this:
Rich (BB code):
ASUSWRT-Merlin RT-AX86U 386.3_2 Fri Aug 6 21:47:27 UTC 2021

I couldn't find any message like that, not in any of the syslogs either. The closest info was in webs_update.log:
Code:
[FWUPDATE][firmware_check_update_main:(361)]---- current version : RT-AC86U 3004 384 82072----
also:
Code:
[FWUPDATE][firmware_check_update_main:(438)]---- download real release note https://dlcdnets.asus.com/pub/ASUS/wireless/ASUSWRT/RT-AC86U_3004_386_44470-g53bccb8_US_note.zip ----
[FWUPDATE][firmware_check_update_main:(449)]---- firmware check update finish ----
So I'm guessing it saw that a newer v386 was available, but the current version is still v384.

What about AiDisk or AiCloud?

Yes, AiDisk was on, which in hindsight was probably a dumb idea :confused:
 

ColinTaylor

Part of the Furniture
It looks like the next time you have physical access to the router you should do a factory reset with initialize and configure your router again. :(
 

Ohia

Occasional Visitor
OpenVPN shouldn`t be executing that file, and I know at least one malware which installs itself into both openvpn-event and /etc/profile .
Here is my /etc/profile:
Code:
export PATH="/bin:/usr/bin:/sbin:/usr/sbin:/home/$USER:/mmc/sbin:/mmc/bin:/mmc/u
sr/sbin:/mmc/usr/bin:/opt/sbin:/opt/bin:/opt/usr/sbin:/opt/usr/bin"             
export PS1='\[email protected]\h:\w\$ '                                                       
export LD_LIBRARY_PATH=/lib:/usr/lib:/lib/aarch64                               
                                                                                
alias l='ls -lFA'                                                               
alias ll='ls -lF'                                                               
                                                                                
ldd() {                                                                         
        LD_TRACE_LOADED_OBJECTS=1 $*;                                           
}                                                                               
                                                                                
[ -n "${TMOUT+x}" ] || export TMOUT="$(nvram get shell_timeout 2>/dev/null)"   
                                                                                
[ -f /jffs/etc/profile ] && . /jffs/etc/profile                                 
[ -f /opt/etc/profile ] && . /opt/etc/profile
Neither /jffs/etc/profile nor /opt/etc/profile exist.
Also, my /jffs/etc/ directory is empty.

Does this mean I'm in the clear....?
 

ColinTaylor

Part of the Furniture
Does this mean I'm in the clear....?
Same malware reported in this thread. See the response from Asus. Understandably they don't go into the technical details of how it works.

Even though the malware is being blocked there's still something not right with your firmware because it shouldn't be attempting to run that script when starting the VPN. So you need to factory reset it as soon as possible, if only to have a working VPN server again.
 

Ohia

Occasional Visitor
Same malware reported in this thread. See the response from Asus. Understandably they don't go into the technical details of how it works.

Even though the malware is being blocked there's still something not right with your firmware because it shouldn't be attempting to run that script when starting the VPN. So you need to factory reset it as soon as possible, if only to have a working VPN server again.
Yikes. OK. Time to use those air miles.... thanks for your responses.
 

RMerlin

Asuswrt-Merlin dev
Does this mean I'm in the clear....?
Asus' security daemon probably cleaned it up once it was discovered. However it might not have reverted config changes done by it, such as whatever was running that profile script (which initially contained the payload).

At the very least I recommend rebooting the router in case there's still an active process. Doing a factory default reset (including wiping the JFFS partition) whenever possible is strongly recommended.
 

Ohia

Occasional Visitor
It looks like the next time you have physical access to the router you should do a factory reset with initialize and configure your router again. :(

So... I managed to installed a Ubuntu VM on my NAS and now have browser GUI access to my router :p I should be able to do most of the cleanup this way... can't do a full factory reset since I'm worried it'll kill the NAS's internet connection, which is the only lifeline I have right now.
I did notice some Custom Scripting at the bottom of my VPN config:
Code:
up "/bin/sh /jffs/etc/profile"
script-security 3
Is there any way this could not be malware?
 

RMerlin

Asuswrt-Merlin dev
So... I managed to installed a Ubuntu VM on my NAS and now have browser GUI access to my router :p I should be able to do most of the cleanup this way... can't do a full factory reset since I'm worried it'll kill the NAS's internet connection, which is the only lifeline I have right now.
I did notice some Custom Scripting at the bottom of my VPN config:
Code:
up "/bin/sh /jffs/etc/profile"
script-security 3
Is there any way this could not be malware?
The first line makes absolutely no sense in a normal setup, and the second one allows passwords to be passed through an environment variable - so possibly stealing or bypassing an OpenVPN password.
 

ColinTaylor

Part of the Furniture
The first line makes absolutely no sense in a normal setup, and the second one allows passwords to be passed through an environment variable - so possibly stealing or bypassing an OpenVPN password.
As the firmware was version 384 until recently I'd speculate that rogue script could also have stolen other user account names and passwords that were stored on the router as it predates them being encrypted (IIRC).

So @Ohia needs to change all such usernames and/or passwords that can be used remotely.
 

Ohia

Occasional Visitor
Thanks for your help, everyone. I was able to fully reset my OpenVPN settings thanks to RMerlin and Draco's excellent advice in this thread. Changed my login/password and implemented the security tips on ASUS' page.
I am now able to OpenVPN in again, thank $DEITY. Will keep an eye on any strange happenings and next time I'm on site I'll flash to the latest v386 firmware. Still mystified as to what vector this malware could have used...
 

Ohia

Occasional Visitor
As the firmware was version 384 until recently I'd speculate that rogue script could also have stolen other user account names and passwords that were stored on the router as it predates them being encrypted (IIRC).
Thanks, exactly what kind of usernames/passwords could these be, other than OpenVPN? Wireless I guess...
 

ColinTaylor

Part of the Furniture
Thanks, exactly what kind of usernames/passwords could these be, other than OpenVPN? Wireless I guess...
Probably just VPN. But I'm thinking there might be other things you used on the router like DDNS email addresses and passwords, AiCloud logins, or other kinds of account information that you might use elsewhere (e.g. Amazon login).
 

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top