OpenVPN - estimate performance via OpenVPN

[sfx2000]

Been tinkering around a bit more with OpenVPN, and seeing how to estimate performance across the link... found this one over on the pfSense forums, and it seems to scale pretty well...

In the past, folks were testing OpenSSL speeds, but that's only one part of OpenVPN, and doesn't take into account the OpenVPN application, or the HMAC...

So for comparing OpenVPN performance, I have started using this benchmark instead:

openvpn --genkey --secret /tmp/secret
time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc

This generates a temporary key (it won't mess with your primary key configs), and then asks OpenVPN to run a shedload of packets thru the OpenVPN app - see the note below...

Then to give the execution time in seconds a real-world meaning:

( 3200 / execution_time_seconds ) = Projected Maximum OpenVPN Performance in Mbps

The projected speed should be an upper limit under optimum conditions...

The magic number of 3200 comes from summing 1 to 20000, multiply by 2 for encrypt and decrypt and by 8 bits/byte and divide by 1,000,000 for a result of Mbps

so for an intel J1800 (Baytrail-D @ 2.41GHz) on Ubuntu 16.04LTS

sfx@blaster:~$ openvpn --genkey --secret /tmp/secret
sfx@blaster:~$ time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc

real    0m18.405s
user    0m18.420s
sys    0m0.012s

This gives us;

3200/18.405 = 173.87 Mbps max thruput over OpenVPN

Compare notes across other platforms?

 

[john9527]

Was playing around and ran this on an AC68P overclocked to 1200MHz (I added the time command to my fork, it's not standard)....

admin@AC68P-06650:/tmp/home/root# /jffs/scripts/openvpnperf.sh
aes-128-cbc results
real    0m 45.24s
user    0m 42.86s
sys     0m 2.33s
aes-256-cbc results
real    0m 48.57s
user    0m 45.98s
sys     0m 2.53s

So for
aes-128-cbc, 3200/45.24 = 70.73 Mbps
aes-256-cbc, 3200/48.57 = 65.88 Mbps

My gut feel from actual results was that the max was in the low-mid 60's, so it's close. Maybe just a bit optimistic.

 

[sfx2000]

And to link the two threads together - this was the OpenSSL numbers thread...

http://www.snbforums.com/threads/req-need-some-openssl-numbers-info.31444/

OpenSSL performance is important, but these numbers are more relevent for real-world expectations of what an end-point _can_ do... it's up to the broadband connection after that...

Comments - respond here, as this is more OpenVPN oriented, and useful for those who are looking to get the most out of their OVPN solution...

 

[sfx2000]

aes-128-cbc, 3200/45.24 = 70.73 Mbps
aes-256-cbc, 3200/48.57 = 65.88 Mbps

Suggests that Tribal Knowledge, e.g. AES-128 is much faster than AES-256 - looks like it is not - so to be more secure, it's not that much overhead...

 

[john9527]

Suggests that Tribal Knowledge, e.g. AES-128 is much faster than AES-256 - looks like it is not - so to be more secure, it's not that much overhead...

Agreed....and matches my 'real world' observations. Could be due to OpenSSL having optimized assembler for AES ARM support.

For me the performance killer is the auth digest. I normally run with auth SHA1. Again, real world, changing to auth SHA256 results in about a 40-50% throughput hit.

 

[maurer]

i've seen this benchmarking first on pfsense forums and i've noticed that openvpn-polarssl (since supported 1.3.x) got up to 70% boost for mipsel boxes.
so for openvpn on rt-n/ac/16/66 (MIPS based) I recommend to run entware's openvpn-polarssl

 

[CiscoX]

[2.3.2-RELEASE][xxxxxxxxxxxxxxxxxxxxx]/xxxxxx: time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc

9.454u 0.425s 0:09.88 99.8%  742+178k 0+0io 0pf+0w

This gives me:
3200/9.88 = 323.88 Mbps

 

[Xentrk]

Here are my results

aes-256-cbc:  3200/9.33 = 342.97 Mbps
aes-128-cbc:  3200/9.18 = 348.58 Mpbs
aes-128-gcm: 3200/8.54 = 374.70 Mbps

 

[sfx2000]

Here are my results

aes-256-cbc:  3200/9.33 = 342.97 Mbps
aes-128-cbc:  3200/9.18 = 348.58 Mpbs
aes-128-gcm: 3200/8.54 = 374.70 Mbps

Fun stuff - playing around with a Kaby Lake NUC - i5-7260... Should get around wired speed on a gigabit connection...

aes-256-gcm: 3200/2.99 = 1070
aes-128-gcm: 3200/2.95 = 1084
aes-256-cbc: 3200/3.59 = 891
aes-128-cbc: 3200/3.52 = 909

 

[RMerlin]

Something changed with OpenSSL 1.1.x BTW, I get 3 seconds regardless of whether it runs on a Cortex B53 or an Intel i7 7700K...

 

[sfx2000]

Something changed with OpenSSL 1.1.x BTW, I get 3 seconds regardless of whether it runs on a Cortex B53 or an Intel i7 7700K...

That's odd - still works fine here...

sfx@jetson:~$ openssl version && openvpn --genkey --secret /tmp/secret && time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-cbc
OpenSSL 1.1.0g  2 Nov 2017
Thu Jun  6 19:32:37 2019 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode

real 0m9.734s
user 0m9.676s
sys 0m0.028s

Jetson Nano seems good for 328.74 Mb/Sec on the latest Jetpack (L4T) - it's all core, as the Crypto element in TX1 was specifically disabled due to lack of documentation...

Here's AES-128-GCM...

sfx@jetson:~$ openssl version && openvpn --genkey --secret /tmp/secret && time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-gcm
OpenSSL 1.1.0g  2 Nov 2017
Thu Jun  6 19:38:38 2019 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode

real 0m9.055s
user 0m9.028s
sys 0m0.004s

Jetson Nano isn't the fastest Cortex-A57, but it does well enough...

sfx@jetson:~$ lscpu
Architecture:        aarch64
Byte Order:          Little Endian
CPU(s):              4
On-line CPU(s) list: 0-3
Thread(s) per core:  1
Core(s) per socket:  4
Socket(s):           1
Vendor ID:           ARM
Model:               1
Model name:          Cortex-A57
Stepping:            r1p1
CPU max MHz:         1428.0000
CPU min MHz:         102.0000
BogoMIPS:            38.40
L1d cache:           32K
L1i cache:           48K
L2 cache:            2048K
Flags:               fp asimd evtstrm aes pmull sha1 sha2 crc32

And the classic OpenSSL numbers for aes-128-cbc, aes-256-cbc, bf-cbc (this should really be deprecated these days if one is looking at potential OpenVPN performance, as the numbers there are quite silly and don't line up with the OpenVPN test on this thread)

sfx@jetson:~$ openssl speed aes-128-cbc aes-256-cbc bf-cbc
Doing aes-128 cbc for 3s on 16 size blocks: 11705629 aes-128 cbc's in 2.99s
Doing aes-128 cbc for 3s on 64 size blocks: 3103429 aes-128 cbc's in 3.00s
Doing aes-128 cbc for 3s on 256 size blocks: 802425 aes-128 cbc's in 2.99s
Doing aes-128 cbc for 3s on 1024 size blocks: 202150 aes-128 cbc's in 2.99s
Doing aes-128 cbc for 3s on 8192 size blocks: 25329 aes-128 cbc's in 3.00s
Doing aes-128 cbc for 3s on 16384 size blocks: 12656 aes-128 cbc's in 2.99s
Doing aes-256 cbc for 3s on 16 size blocks: 8958206 aes-256 cbc's in 2.99s
Doing aes-256 cbc for 3s on 64 size blocks: 2341683 aes-256 cbc's in 3.00s
Doing aes-256 cbc for 3s on 256 size blocks: 600410 aes-256 cbc's in 2.99s
Doing aes-256 cbc for 3s on 1024 size blocks: 150961 aes-256 cbc's in 2.99s
Doing aes-256 cbc for 3s on 8192 size blocks: 18875 aes-256 cbc's in 3.00s
Doing aes-256 cbc for 3s on 16384 size blocks: 9447 aes-256 cbc's in 2.99s
Doing blowfish cbc for 3s on 16 size blocks: 9917261 blowfish cbc's in 2.99s
Doing blowfish cbc for 3s on 64 size blocks: 2669863 blowfish cbc's in 3.00s
Doing blowfish cbc for 3s on 256 size blocks: 679434 blowfish cbc's in 2.99s
Doing blowfish cbc for 3s on 1024 size blocks: 168231 blowfish cbc's in 3.00s
Doing blowfish cbc for 3s on 8192 size blocks: 21082 blowfish cbc's in 2.99s
Doing blowfish cbc for 3s on 16384 size blocks: 10559 blowfish cbc's in 3.00s
OpenSSL 1.1.0g  2 Nov 2017
built on: reproducible build, date unspecified
options:bn(64,64) rc4(char) des(int) aes(partial) blowfish(ptr) 
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/aarch64-linux-gnu/engines-1.1\"" 
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
blowfish cbc     53068.96k    56957.08k    58172.28k    57422.85k    57760.45k    57666.22k
aes-128 cbc      62638.82k    66206.49k    68702.61k    69231.30k    69165.06k    69349.80k
aes-256 cbc      47936.89k    49955.90k    51406.34k    51700.36k    51541.33k    51765.77k