What's new
  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

ASUS RT-AX86U OpenVPN Server Error - Key Too Small

dsneed

Occasional Visitor
I just setup my ASUS RT-AX86U, replacing an RT-AC86U. I keep getting an error when trying to connect on my iPhone (the only device I use it) using the OpenVPN app.

CORE_THREAD_ERROR OpenSSLContext:
SSL_CTX_use_certificate failed:
error:0A00018F:SSL routines::ee key too small

Not sure how to resolve this issue. Please help

client
dev tun
proto tcp-client
remote <server> <port>
resolv-retry infinite
nobind
float
ncp-ciphers AES-256-GCM:AES-128-GCM:AES-256-CBC:AES-128-CBC:CHACHA20-POLY1305
tls-cipher TLS-DHE-RSA-WITH-AES-128-CBC-SHA
auth SHA256
keepalive 15 60
auth-user-pass
remote-cert-tls server
<ca>
-----BEGIN CERTIFICATE-----

-----END CERTIFICATE-----

</ca>
<cert>
-----BEGIN CERTIFICATE-----

-----END CERTIFICATE-----

</cert>
<key>
-----BEGIN PRIVATE KEY-----

-----END PRIVATE KEY-----

</key>
<tls-auth>
-----BEGIN OpenVPN Static key V1-----

-----END OpenVPN Static key V1-----

</tls-auth>
key-direction 0
 
Last edited:
If this is over wifi it won't work. Connection needs to be made over the internet.
 
Not sure how to resolve this issue.

Default Asuswrt OpenVPN Server configuration works with no issues to OpenVPN Connect app on iOS. No manual configuration is needed except if one wants to change the server port and tell the VPN server what to route to the clients - LAN only or LAN + Internet. Check your configuration and test from Internet only as indicated above.
 
If the Security Level is left at the default Preferred I get the error on the left. If I change it to Legacy, the same profile works. Something has to be missing to get the profile to work with the Preferred security level.

Clipboard_04-15-2024_01.png
Clipboard_04-15-2024_03.png
 
Perhaps OpenVPN Connect app wants to see stronger encryption.

On the router change HMAC Authentication from SHA1 to SHA256 and try again. You can also disable Compression, if enabled.
 
Last edited:
Has this been resolved? I’m getting the same issue
Ran into the same problem. All I had to do was select RSA Encryption: 2048 bit when I first configured my VPN server, then it worked without issue.

If you don't see this option, reset your server to default, then it should show up.
1000001683.png
 
i have the same issue legacy mode works
i have rt-ax88u, seems that RSA encryption mode 2048 is not there anymore, its also not under the advanced settings

1760984969378.png
 
i have the same issue legacy mode works
i have rt-ax88u, seems that RSA encryption mode 2048 is not there anymore, its also not under the advanced settings

View attachment 68478
The setting only appear when first setting up the server. You will need to reset it to its default and reconfigure it anew to change the key strength. Would be the perfect time to also upgrade to a newer cipher and disable encryption if you were still using it (encryption is a security risk, and is being phased out by OpenVPN).
 

Support SNBForums w/ Amazon

If you'd like to support SNBForums, just use this link and buy anything on Amazon. Thanks!

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Back
Top