Using the Cloud for Better Site-to-Site Speed

[Samir]

I have several site-to-site VPNs. I connect to our main hub and access the network there via rdp over one of those site-to-site links.

However, the bandwidth isn't enough for me to complete all the work remotely, so about every quarter I have to fly to the main site, work locally, and fly back.

I've done some iperf tests and it seems there's not enough direct bandwidth between the two sites to work as fast as I want. And this is even though both sites have healthy Internet connections (25/5 and 17/3).

I've looked into Peplink and other VPN bonding products and they are just out of the price range.

I've looked into routers that could possibly bond VPN connections, but it's going to take me a month to get that up and running solid.

Since most cloud providers have regional virtual data centers, and I think a connection to one of those would be at full bandwidth. I have an idea to try connecting these two sites via a cloud service. But I want to know if someone has tried this before.

Has anyone tried connecting to a cloud service via a site-to-site connection from two different locations and using the cloud as the backhaul as opposed to the Internet? What results have you gotten? Faster? Slower? Same?

Thank you in advance for any replies.

 

[Motofixxer]

I haven't tried it but it seems like the newer cloud storage options would be a very viable possibility. With the encryption options pretty much as a default with the better services, it seems to be a rather easy task. It certainly would be an easy test to gauge performance.

 

[Samir]

Thank you for the reply. I've found a few companies that provide saas to do this, but I've been able to up the bandwidth at the source so maybe I can solve my problem that way.

 

[System Error Message]

Unless you're using cloud storage as a buffer i dont see how you can improve speeds using a middle man other than caching but if your passing files through and forth caching isnt going to help. However if the route from you to the VPN server to your work is better than from you to your workplace than it may be worth it.

As you have seen in my other thread you can load balance using VPN with mikrotik to get more bandwidth. I havent checked pfsense yet but you could try that virtually too to see if it supports it or has a plugin that does.

Windows is horrible at bonding normally but windows server supports NIC teaming and load balancing so it may be able to do what you want but a linux server is usually better at that. Still these are some options you can use either locally or on a VPN server to bond multiple links. You dont necessarily have to bond multiple links to get more bandwidth when you can load balance to do the same thing. The only difference is that bonding would usually be done on L2 or on the physical layer while load balancing is done on the upper layers. Basically bonding just reduces resource usage normally but if you have the CPU and RAM it is not necessary.

If you want to connect 2 sites through a datacenter you will need to either rent a VPN server or a Virtual private server and set it up yourself. There are VPN services that would fit your need. Try to see if they have a demo so you can check how fast the route is between you, the server and your workplace (iperf wont do on demos, you would need traceroutes and pings). If your workplace is the VPN server(and you cant change that) than you will need a virtual private server as a VPN server and router.

If your workplace is the server you may not be able to instigate multiple VPN connections using the same account. It really depends on the VPN server.

Edit: What peplink does isnt actually "bonding" interfaces but it load balances them using algorithms and such that it appears to be bonded. The same can be achieved with other setups but it takes more work and thinking so paying more for peplink is for the convenience and support.

 

[Samir]

Unless you're using cloud storage as a buffer i dont see how you can improve speeds using a middle man other than caching but if your passing files through and forth caching isnt going to help. However if the route from you to the VPN server to your work is better than from you to your workplace than it may be worth it.

As you have seen in my other thread you can load balance using VPN with mikrotik to get more bandwidth. I havent checked pfsense yet but you could try that virtually too to see if it supports it or has a plugin that does.

Windows is horrible at bonding normally but windows server supports NIC teaming and load balancing so it may be able to do what you want but a linux server is usually better at that. Still these are some options you can use either locally or on a VPN server to bond multiple links. You dont necessarily have to bond multiple links to get more bandwidth when you can load balance to do the same thing. The only difference is that bonding would usually be done on L2 or on the physical layer while load balancing is done on the upper layers. Basically bonding just reduces resource usage normally but if you have the CPU and RAM it is not necessary.

If you want to connect 2 sites through a datacenter you will need to either rent a VPN server or a Virtual private server and set it up yourself. There are VPN services that would fit your need. Try to see if they have a demo so you can check how fast the route is between you, the server and your workplace (iperf wont do on demos, you would need traceroutes and pings). If your workplace is the VPN server(and you cant change that) than you will need a virtual private server as a VPN server and router.

If your workplace is the server you may not be able to instigate multiple VPN connections using the same account. It really depends on the VPN server.

Edit: What peplink does isnt actually "bonding" interfaces but it load balances them using algorithms and such that it appears to be bonded. The same can be achieved with other setups but it takes more work and thinking so paying more for peplink is for the convenience and support.

A cloud provider can do a site-to-site vpn to my sites and then connect them on their MPLS backbone, bypassing any general Internet latency. It's not cheap, but it was a solution I was looking into.

Neither mikrotik or pfsense support vpn load balancing. I can't recall which one (or both), but there is a bridge mode that could be used to join the networks. However, I haven't seen anyone that has had bandwidth increasing success with that with multiple wans in the same way that peplinks speedfusion works.

I know there's different ways to do it, but the problem is which way will truly be reliable day-in and day-out. Software solutions can have bugs and issues (just like hardware can), and if they were solid solutions, you'd see them more in enterprise networks.

Peplink will actually spread the packets across all the interfaces and then reassemble them on the other end. But it takes some software to do this as the packets could come in at different times due to latency. I've talked to them about their solution and it's pretty solid, but also more expensive than I can afford.

 

[System Error Message]

the load balancing i did with mikrotik across multiple PPPOE worked well with minimal configuration. It was however multiple links to the same ISP to improve my internet speed (the VDSL had to travel 2KM) and packets did use multiple links like they were bonded but packets would go through the best routes available at each time. Im not saying you should try mikrotik, im saying that using a service may be better if your connection isnt the problem. Using multiple connections and load balancing i managed to get better throughput.

Ive read many who use mikrotik that set up multiple links load balance between them instead of bonding and it does work well. The only issue with routerOS is that each link uses only a single core so those who bought the CCR1036 would create like 30 tunnels or links to fully utilise the device and actually do get multi gigabit throughput with many tunnels, VPN and links. They did mention configuring load balancing for that many links to be a pain with the amount of configuration required but if you're only doing 2 links and you cant afford peplink than it might be worth looking at load balancing instead.

 

[Samir]

the load balancing i did with mikrotik across multiple PPPOE worked well with minimal configuration. It was however multiple links to the same ISP to improve my internet speed (the VDSL had to travel 2KM) and packets did use multiple links like they were bonded but packets would go through the best routes available at each time. Im not saying you should try mikrotik, im saying that using a service may be better if your connection isnt the problem. Using multiple connections and load balancing i managed to get better throughput.

Ive read many who use mikrotik that set up multiple links load balance between them instead of bonding and it does work well. The only issue with routerOS is that each link uses only a single core so those who bought the CCR1036 would create like 30 tunnels or links to fully utilise the device and actually do get multi gigabit throughput with many tunnels, VPN and links. They did mention configuring load balancing for that many links to be a pain with the amount of configuration required but if you're only doing 2 links and you cant afford peplink than it might be worth looking at load balancing instead.

I read alot about the mikrotik products and even the forum, but never ran across a successful implementation of what I needed.

If you've got some links to read, I'll check it out again.

 

[System Error Message]

There are no examples because they all cater to more complicated networks. I figured out your problem now. You want to use remote desktop over multiple VPN connections to your office computer. There is also another protocol called RDP i think but thats for something else.

The way routerOS was made was that it is dynamic when it comes to routing which is what i like about it. You can assign as many IPs and networks as you like to a single interface or the same IP to multiple interface and have it still work. Your problem is that you only want to use remote desktop but expect routers to already know the protocol which isnt possible because certain things are closed sourced or licensed. Skype was closed source but someone managed to reverse engineer the protocol and we now have an L7 hash that we can use to prioritise VOIP or even block skype file transfers. To routerOS VPN is just another route and i would suggest using simple tricks like getting the L7 hash for RDP or simply getting the port and making a rule with a rate limit and another rule with no limit to get the connections over 2 VPNs. The important thing here isnt that the router does what you want for remote desktops but that both computers see only 1 gateway/device. That means if you wanted to use peplink than both ends need to use peplink.

The simplest way to do this in routerOS is simply to create a firewall rule based on destination port/protocol with a rate limit in NAT/filter to go through the VPN and than another similar rule with no rate limit to catch everything else to go through the other VPN. You just set the rate limit in bandwidth to match your VPN link speed that you would expect to get. Because this is a site to site VPN in routerOS just configure the routes (2 of the same routes with the same weights) and it will load balance between them equally if they have the same latency and assign the same ip address for both interfaces. Its a bit hackish but with routerOS it is possible but requires a network engineer to understand it better. The learning curve for routerOS can sometimes be steep but it tends to be worth it. I just found out that my mikrotik CRS is capable of firewall on the switch chip itself but the terminology and configuration is very different to that of a different fully managed switch and they're still adding more functionality.

 

[Samir]

There are no examples because they all cater to more complicated networks. I figured out your problem now. You want to use remote desktop over multiple VPN connections to your office computer. There is also another protocol called RDP i think but thats for something else.

The way routerOS was made was that it is dynamic when it comes to routing which is what i like about it. You can assign as many IPs and networks as you like to a single interface or the same IP to multiple interface and have it still work. Your problem is that you only want to use remote desktop but expect routers to already know the protocol which isnt possible because certain things are closed sourced or licensed. Skype was closed source but someone managed to reverse engineer the protocol and we now have an L7 hash that we can use to prioritise VOIP or even block skype file transfers. To routerOS VPN is just another route and i would suggest using simple tricks like getting the L7 hash for RDP or simply getting the port and making a rule with a rate limit and another rule with no limit to get the connections over 2 VPNs. The important thing here isnt that the router does what you want for remote desktops but that both computers see only 1 gateway/device. That means if you wanted to use peplink than both ends need to use peplink.

The simplest way to do this in routerOS is simply to create a firewall rule based on destination port/protocol with a rate limit in NAT/filter to go through the VPN and than another similar rule with no rate limit to catch everything else to go through the other VPN. You just set the rate limit in bandwidth to match your VPN link speed that you would expect to get. Because this is a site to site VPN in routerOS just configure the routes (2 of the same routes with the same weights) and it will load balance between them equally if they have the same latency and assign the same ip address for both interfaces. Its a bit hackish but with routerOS it is possible but requires a network engineer to understand it better. The learning curve for routerOS can sometimes be steep but it tends to be worth it. I just found out that my mikrotik CRS is capable of firewall on the switch chip itself but the terminology and configuration is very different to that of a different fully managed switch and they're still adding more functionality.

I figured out how to do that, but the problem is that this limits the maximum speed to the maximum speed of a single link. Peplink's bonding will actually increase the vpn link's speed by taking the packets in the rdp session and span them across both link and reassemble them on the other side, effectively increasing the overall speed to the total of all the wans.

 

[System Error Message]

That is what i just mentioned, to send the packets through both links but by using the first link till it is full and than sending everything else through the 2nd link. If you're using end to end links on the router than it will work fine because the PCs will only see 1 gateway/system.

How does this limit things to the maximum speed of a single link when packets would be using both links at the same time? This is a form of bonding done manually which uses the first link until it is full before sending packets across the 2nd link but without tolerance for any link disconnecting.

 

[Samir]

That is what i just mentioned, to send the packets through both links but by using the first link till it is full and than sending everything else through the 2nd link. If you're using end to end links on the router than it will work fine because the PCs will only see 1 gateway/system.

How does this limit things to the maximum speed of a single link when packets would be using both links at the same time? This is a form of bonding done manually which uses the first link until it is full before sending packets across the 2nd link but without tolerance for any link disconnecting.

I guess this would be one way to do it, but you're effectively having to fill up the first pipe before the second is used.

 

[System Error Message]

you could always add a few rules and mess with the logic to get what you want but i just mentioned the simplest way you could achieve what you want. Not sure if it is the RDP you use but routerOS has RDP as one of the predefined protocols in firewall. If you go to demo.mt.lv and navigate to firewall and add new you will see RDP as one of the predefined protocols.

Perhaps you should give it a go. Any RB with a fast MIPS would do well but if you want you can go for the PPC based one or even the TILE. A fast MIPS would do fine for 50Mb/s of VPN while a 1 Ghz PPC would do up to 500Mb/s of VPN per core while each TILE core does 300Mb/s of VPN. MIPS based RBs are very inexpensive and the fast ones will be faster in VPN throughput compared to the lowest end peplink while costing a fraction of the price. When choosing a routerboard the only difference between them are hardware features and speed. All of them run routerOS that has the same features except for the switch which is different for every routerboard. Even the old RB450G is sufficient for your speeds. RouterOS supports MPLS routing too.

For your setup to work both sides must be able to set up site to site VPN and must be able to configure load balancing however if you only want the upload bandwidth than only the upload side needs to do load balancing for sending packets

Ubiquiti might be able to do the same thing (see their manual) and their edgerouter lite costs $80 which is very fast for NAT but i am not sure about their VPN speeds. The edgerouter lite uses a dual core MIPS at 500Mhz and VPN is most likely going to utilise a single core per link. For mikrotik the closest thing to the performance would be the RB 850Gx2 which has a dual core PPC at 500Mhz but has much faster VPN performance and lower penalty when it comes to firewall and costs about $120 with the chassis and plug.

 

[Samir]

you could always add a few rules and mess with the logic to get what you want but i just mentioned the simplest way you could achieve what you want. Not sure if it is the RDP you use but routerOS has RDP as one of the predefined protocols in firewall. If you go to demo.mt.lv and navigate to firewall and add new you will see RDP as one of the predefined protocols.

Perhaps you should give it a go. Any RB with a fast MIPS would do well but if you want you can go for the PPC based one or even the TILE. A fast MIPS would do fine for 50Mb/s of VPN while a 1 Ghz PPC would do up to 500Mb/s of VPN per core while each TILE core does 300Mb/s of VPN. MIPS based RBs are very inexpensive and the fast ones will be faster in VPN throughput compared to the lowest end peplink while costing a fraction of the price. When choosing a routerboard the only difference between them are hardware features and speed. All of them run routerOS that has the same features except for the switch which is different for every routerboard. Even the old RB450G is sufficient for your speeds. RouterOS supports MPLS routing too.

For your setup to work both sides must be able to set up site to site VPN and must be able to configure load balancing however if you only want the upload bandwidth than only the upload side needs to do load balancing for sending packets

Ubiquiti might be able to do the same thing (see their manual) and their edgerouter lite costs $80 which is very fast for NAT but i am not sure about their VPN speeds. The edgerouter lite uses a dual core MIPS at 500Mhz and VPN is most likely going to utilise a single core per link. For mikrotik the closest thing to the performance would be the RB 850Gx2 which has a dual core PPC at 500Mhz but has much faster VPN performance and lower penalty when it comes to firewall and costs about $120 with the chassis and plug.

I'm sure any of them can handle the vpn connection as we don't even have that much bandwidth. And that's the point of adding a second wan connection and then also routing the rdp on that as well, so each packet will take the fastest path, effectively doubling the speed.

I'll check out the demo when I get time as that's going to really give me a feel for it.