386.1_2: openvpn-event causes “Linux route add command failed”

  • ATTENTION! As of November 1, 2020, you are not able to reply to threads 6 months after the thread is opened if there are more than 500 posts in the thread.
    Threads will not be locked, so posts may still be edited by their authors.
    Just start a new thread on the topic to post if you get an error message when trying to reply to a thread.

Spud

Occasional Visitor
Hi everyone,

I just updated my AC86U to Merlin 386.1_2 (from 386.1, full reset and manual setup). I use an openvpn-event script to bypass the VPN for Plex remote access. Since the update, it’s causing the VPN client to throw errors. The VPN client launches but the routing doesn’t work. Here’s a paste from the system log:


FF65F042-B47D-436B-AFE4-BA682DB6BC5E.jpeg



I’m using this openvpn-event script:


459C6901-A687-446E-A0AD-29154396FEE7.jpeg



Does anyone know what the problem might be? Thanks for your ideas.
 

eibgrad

Very Senior Member
Not 100% sure, but the errors are NOT from your script per se, but from the router attempting to add VPN routes. That can happen sometimes if there's more than one attempt to apply the routes (i.e., they already exist).

The one thing you definitely should NOT be doing in any OpenVPN event script is using a sleep command! OpenVPN will only provide a small window of opportunity for the called event to complete. And if it doesn't or takes too long, it will kill that subprocess, and either try again, or eventually exit OpenVPN. And that may be what's happening here. Try getting rid of the sleep command. It's completely unnecessary anyway.
 
Last edited:

eibgrad

Very Senior Member
BTW, those failed route commands *appear* to be attempts to add specific hosts (x.x.x.x/32) to the WAN rather than the VPN. I wouldn't normally expect to see that except for perhaps the server's IP address since obviously that needs to always remain bound to the WAN. But I'm seeing attempts to add *multiple* hosts, as if you perhaps have your own route directives in custom config. Could that be the case? And if so, could we see specifically what you added?
 

Spud

Occasional Visitor
BTW, those failed route commands *appear* to be attempts to add specific hosts (x.x.x.x/32) to the WAN rather than the VPN. I wouldn't normally expect to see that except for perhaps the server's IP address since obviously that needs to always remain bound to the WAN. But I'm seeing attempts to add *multiple* hosts, as if you perhaps have your own route directives in custom config. Could that be the case? And if so, could we see specifically what you added?

Thanks for your help, eibgrad. I’ve removed sleep as you suggested, but no change.

My custom config is below. It automatically populates when adding the NordVPN opvn file, except the last line to route Plex server. Strangely, I’ve been using the same solution for many months, but only now it’s broken.


resolv-retry infinite
remote-random
tun-mtu 1500
tun-mtu-extra 32
mssfix 1450
ping 15
ping-restart 0
ping-timer-rem
remote-cert-tls server
pull
fast-io
cipher AES-256-CBC
route plex.tv 255.255.255.255 net_gateway
 
Last edited:

eibgrad

Very Senior Member
Then it as I suspected, a route directive that's producing the errors. But I don't know why it would be failing. It obviously resolved plex.tv to several public IPs. And the net_gateway reserved word gets automatically converted into the WAN's default gateway by OpenVPN. So it certainly *looks* correct.

I tried the same route directive on my own OpenVPN client (ExpressVPN) and it worked fine. But I'm still using 384.19. Maybe something changed w/ the latest build.
 

Spud

Occasional Visitor
Then it as I suspected, a route directive that's producing the errors. But I don't know why it would be failing. It obviously resolved plex.tv to several public IPs. And the net_gateway reserved word gets automatically converted into the WAN's default gateway by OpenVPN. So it certainly *looks* correct.

I tried the same route directive on my own OpenVPN client (ExpressVPN) and it worked fine. But I'm still using 384.19. Maybe something changed w/ the latest build.

Earlier, when I upgraded to 386.1, there was no issue. Something appears to have changed with 386.1_2. Maybe I can do a complete reset and revert.
 

eibgrad

Very Senior Member
P.S. I just noticed a difference. My build is using /sbin/route (the old way), but you're using /usr/sbin/ip (the more modern version).

Code:
Mar  9 09:09:04 ovpn-client1[30910]: /sbin/route add -net 108.128.10.254 netmask 255.255.255.255 gw 192.168.63.1
Mar  9 09:09:04 ovpn-client1[30910]: /sbin/route add -net 99.81.153.144 netmask 255.255.255.255 gw 192.168.63.1
Mar  9 09:09:04 ovpn-client1[30910]: /sbin/route add -net 99.81.164.127 netmask 255.255.255.255 gw 192.168.63.1

I wonder if that's the problem. But the syntax on yours looks correct.
 

eibgrad

Very Senior Member
Ok, I noticed something else as well. Using the route command, it works w/ domain names. But using the ip command, it doesn't. The ip command will NOT automatically resolve domain names the way the route command does. But it's still strange that syslog shows the IPs, as if the resolution worked. Not unless that version of OpenVPN is taking care of that operation separately, before applying the routes.

Could just be a red herring, but it *is* an obvious difference between you and me.
 
Last edited:

eibgrad

Very Senior Member
What you might try doing is adding the routes *without* using the domain name. Just to see if it makes a difference (iow, three separate route directives in the custom config field using the explicit IPs).

Code:
route 108.128.10.254 255.255.255.255 net_gateway
route 99.81.153.144 255.255.255.255 net_gateway
route 99.81.164.127 255.255.255.255 net_gateway
 

eibgrad

Very Senior Member
BTW, is there anything else about this config we need to know? I noticed you're using the 5th OpenVPN client instance, which suggests the others are also configured. Are any of those also active at the same time? And if so, are any of these OpenVPN clients NOT using PBR (policy based routing)?
 

Spud

Occasional Visitor
BTW, is there anything else about this config we need to know? I noticed you're using the 5th OpenVPN client instance, which suggests the others are also configured. Are any of those also active at the same time? And if so, are any of these OpenVPN clients NOT using PBR (policy based routing)?

I just tried your suggestion above, but alas no change.

I’m using Client 1 with policy-based routing, that’s all for the moment (I haven’t set up any others after the router reset).
 

eibgrad

Very Senior Member
And so if you remove that plex.tv route directive, everthing otherwise runs normally, no errors? I know that's not what you want, but I'm trying to pin down the culprit here. If necessary, try removing your openvpn-event script too. We need to know exactly what breaks it. Even if that means turning off Routing Policy too. If there are too many variables (route directives, your event script, PBR, etc.), it gets too difficult to know where things are going wrong.
 

Spud

Occasional Visitor
And so if you remove that plex.tv route directive, everthing otherwise runs normally, no errors? I know that's not what you want, but I'm trying to pin down the culprit here. If necessary, try removing your openvpn-event script too. We need to know exactly what breaks it. Even if that means turning off Routing Policy too. If there are too many variables (route directives, your event script, PBR, etc.), it gets too difficult to know where things are going wrong.

Indeed, removing the Plex route directive stops the errors, and the event script runs normally. (Yesterday I removed the script and left the directive - also no errors.)
 

eibgrad

Very Senior Member
And you said you have Routing Policy enabled. Try w/ the route directive specified, your script, but NO Routing Policy. Just left everything default to the VPN.

There has to be something that specifically triggers the errors.
 

Spud

Occasional Visitor
And you said you have Routing Policy enabled. Try w/ the route directive specified, your script, but NO Routing Policy. Just left everything default to the VPN.

There has to be something that specifically triggers the errors.

Sorry I couldn’t respond sooner, thanks very much for your help today.

I’ve now tried the above as well. Again no script errors.

However, in each case so far, Plex initially gives the impression it’s connected (sometimes several minutes before switching offline), but using the app on mobile fails to stream, while Kodi (also via mobile) can’t find the server. Plex server shows the current dynamic WAN IP under Remote Access, but no data seems to flow.

To complicate things, I’ve noticed that the logged error doesn’t necessarily repeat itself under the same circumstances. For example, I put back the policy settings and route command as they were at the start, with the script still in place, and there was no error starting Client 5 a few times.

While the above seems software-related, a comment about my AC86U... I bought an ‘international’ model after being duped that it was UK-specific. My bad. It’s now locked to Europe WiFi and works fine a lot of the time, but it’s always been tricky. For example, switching off one VPN client can kill the connection everywhere or freeze the router for a few minutes. If I reboot it, it often comes back up with no connection and needs a reboot or maybe two. It also has the memory of an elephant, some settings such as device static addresses don’t change after multiple hard resets and downtime. It’s a lot less stable than my old AC56U used to be. Is this at all normal in your experience?
 
Last edited:

eibgrad

Very Senior Member
When it comes to any unreliability in the hardware, or errors that seem to come and go for no rational reason, we're probably at the limits of this forum. It's difficult enough diagnosing problems when things are consistently and predictably wrong, but certain problems might require a hands-on approach.

As far as the behavior of Plex when there are no errors, that's a piece of software I don't use. So I'm no expert on any of its particular quirks. I assume if the VPN is NOT active, it functions normally? Also, what is the intent of the openvpn-event script? It appears to be some form of PBR (policy based routing) of your own, and if that's the case, there's no way for me to understand its impact to the router as whole, or relationship to plex, if any. I don't even know what's in table 254.
 

Spud

Occasional Visitor
When it comes to any unreliability in the hardware, or errors that seem to come and go for no rational reason, we're probably at the limits of this forum. It's difficult enough diagnosing problems when things are consistently and predictably wrong, but certain problems might require a hands-on approach.

As far as the behavior of Plex when there are no errors, that's a piece of software I don't use. So I'm no expert on any of its particular quirks. I assume if the VPN is NOT active, it functions normally? Also, what is the intent of the openvpn-event script? It appears to be some form of PBR (policy based routing) of your own, and if that's the case, there's no way for me to understand its impact to the router as whole, or relationship to plex, if any. I don't even know what's in table 254.

If the VPN is off, Plex works normally via port forwarding.

I used the same method in post #12 to set up the port bypass to WAN for when the VPN is running:

 

eibgrad

Very Senior Member
If the VPN is off, Plex works normally via port forwarding.

I used the same method in post #12 to set up the port bypass to WAN for when the VPN is running:


Thanks for the link. Now I see why I was confused. I didn't realize table 254 was the main table. Usually when referencing tables, at least those well-known, you use the name, if only for reasons of clarity.

Code:
ip rule add fwmark 0x7000/0x7000 table main prio 9990

Anyway, I have a problem w/ something the author of that script has failed to make crystal clear, and it's subtle enough that many users could easily miss it.

Clearly that script is a user defined implementation of PBR (policy based routing). However, as constructed, it *assumes*, in fact *requires*, that Routing Policy be enabled in the OpenVPN client. And that's because when Routing Policy is NOT enabled, the main routing table has its default gateway changed to the VPN by the OpenVPN server (at least for commercial OpenVPN providers). And under such circumstances, pointing your own PBR script back to the main routing table is pointless. But once you enable Routing Policy in the OpenVPN client, that keeps the main routing table pointing to the WAN as its default gateway, and so the script should now work (it may have other issues, but at least as far as this particular issue is concerned).

It would be far better to create your own separate routing table (e.g., 200) and configure it w/ the WAN as its default gateway. That way you're not dependent on how Routing Policy is or isn't configured in the OpenVPN client.

Obviously I don't know if this particular issue accounts for your present problems, but as currently written, the script leaves something to be desired.
 
Last edited:

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top