Wireguard Session Manager - Discussion (3rd) thread

[ZebMcKayhan]

I have being trying to use ipsets and while the routing rules

iptables -t mangle -I PREROUTING -m set --match-set wg11-mac src -j MARK --set-mark 0x1000/0x1000
ip6tables -t mangle -I PREROUTING -m set --match-set wg11-mac src -j MARK --set-mark 0x1000/0x1000

work perfectly and I can run cru to save the ipset the restore command is the problem.

The issue is not the script, rather I cannot get it to run early enough, i.e. before WireGuard so that the wg11-mac value is picked up and restored before wg11-up.sh is run
I have tried running the code in wan-event, nat-start and services-start and (in case loading entware was the issue) moved the saved version from /opt/tmp to /jffs/addons/wireguard. Nothing helps and I always get

ipset list -n wg11-mac
ipset v7.6: The set with the given name does not exist

Once the router is up, then I can re-run any of these (wan-event, nat-start and services-start or a stand alone script) and I then get

ipset list -n wg11-mac
wg11-mac

It may be that the ipset module is busy with loading the Skynet ipsets or just that ipset is not yet functional when wgm starts. I can try disabling Skynet and trying again but for now, I have rebooted the router too many times for everyone's liking and will need to come back to this later.

Nat-start should be early enough but perhaps you need an sleep 10 before it to let other processes finish.

You could also put it in wg11-route-up.sh which is executed just after wg11 is brought up and before rules and routes are put in.

 

[NurseWizzle]

I'm so jealous of everyone that has got this running. I've been reading for a long time about how to get this going and I finally tried jumping in a couple weeks ago. I can't get it. I have tried everything I've read. The only thing I did actually get to work is when I installed some beta firmware to a XT8 and was able to set up a server through the GUI and got my phone to connect. The problem was, for some reason after a bit the firmware would just roll back to stock and no longer would I have access to the GUI. I even installed the beta on my AX88U but for some reason that didn't work.

I don't even know where to begin to troubleshoot. What I'm trying to do is set up a server so I can access my files on my devices from outside my home network; for instance, my computer at work on the work WiFi, I'd like to be able to access the files on my home NAS. Is there anyone that can help?

I've got my AX88U factory reset I believe with the newest stable Merlin. I think I reformatted the USB drive also. That's where I am again.

 

[ZebMcKayhan]

I'm so jealous of everyone that has got this running. I've been reading for a long time about how to get this going and I finally tried jumping in a couple weeks ago. I can't get it. I have tried everything I've read. The only thing I did actually get to work is when I installed some beta firmware to a XT8 and was able to set up a server through the GUI and got my phone to connect. The problem was, for some reason after a bit the firmware would just roll back to stock and no longer would I have access to the GUI. I even installed the beta on my AX88U but for some reason that didn't work.

I don't even know where to begin to troubleshoot. What I'm trying to do is set up a server so I can access my files on my devices from outside my home network; for instance, my computer at work on the work WiFi, I'd like to be able to access the files on my home NAS. Is there anyone that can help?

I've got my AX88U factory reset I believe with the newest stable Merlin. I think I reformatted the USB drive also. That's where I am again.

If you start from scratch with a formatted usb drive you should first install Entware via amtm and then install Wireguard Manager from amtm... then we take it from there. Dont know about the xt8 model but there are many here running AX88U so that shouldn't be a problem.

 

[NurseWizzle]

If you start from scratch with a formatted usb drive you should first install Entware via amtm and then install Wireguard Manager from amtm... then we take it from there. Dont know about the xt8 model but there are many here running AX88U so that shouldn't be a problem.

Thanks for the advice. I think I'll install Entware and then just Wireguard. I was just l trying to install Diversion and a bunch of other things and I don't think that actually broke anything but I dunno.

I did get it working with a GL-iNet router that just showed up today but I'd prefer to get this working on either my AX88U or my Pi.

 

[JGrana]

Thanks for the advice. I think I'll install Entware and then just Wireguard. I was just l trying to install Diversion and a bunch of other things and I don't think that actually broke anything but I dunno.

I did get it working with a GL-iNet router that just showed up today but I'd prefer to get this working on either my AX88U or my Pi.

Its running well on my AX88U. Using wg_manager. It's actually pretty straightforward to setup, Im running both a site-2-site from my home to a cabin and I have a few iPads/iPhones running as clients.

I would recommend you install (via AMTM) Entware and then Wireguard Manager. After you install wg_manager, I would do an update forced to the latest. At the wg_manager prompt:

E:Option ==> uf dev

Then, read through @ZebMcKayhan 's fine Readme:

https://github.com/ZebMcKayhan/WireguardManager#preface

Both @ZebMcKayhan and @Martineau are very responsive to questions/issues as you set it up.

 

[simowdas]

That would explain why you loose contact.

It seems then as parental control is flushing the FORWARD filter Chain without a firewall-start event happens.

Check the syslog if there is any service-event you can use to restart wg when parental control is changed.

Ok. So everything looks ok. In firewall-start there is call to wg_firewall. But wg_firewall calls wg_manager.sh stop and then wg_manager.sh stop.


simo@RT-AX86U-5750:/tmp/home/root# /jffs/addons/wireguard/wg_manager.sh stop

Requesting WireGuard VPN Peer stop (wg21)

Error: Address /jffs/addons/wireguard/wg_server is invalid

*** Usage:
dw/dh/db <physical address in hex> <number>
dw/dh/db <-k> <virtual address in hex> <number>
sw/sh/sb <physical address in hex> <data value1> <data value2> ..<data valueN>
sw/sh/sb <-k> <virtual address in hex> <data value1> <data value2> ..<data valueN>
fw/fh/fb <physical address in hex> <data value> <length>
fw/fh/fb <-k> <virtual address in hex> <data value> <length>
-s (currently works with physical addresses for d*/s*/f* commands
and virtual addresses for s*/f* commands)



simo@RT-AX86U-5750:/tmp/home/root#


Running /jffs/addons/wireguard/wg_server from console works good.

 

[Martineau]

Ok. So everything looks ok. In firewall-start there is call to wg_firewall. But wg_firewall calls wg_manager.sh stop and then wg_manager.sh stop.


simo@RT-AX86U-5750:/tmp/home/root# /jffs/addons/wireguard/wg_manager.sh stop

Requesting WireGuard VPN Peer stop (wg21)

Error: Address /jffs/addons/wireguard/wg_server is invalid

*** Usage:
dw/dh/db <physical address in hex> <number>
dw/dh/db <-k> <virtual address in hex> <number>
sw/sh/sb <physical address in hex> <data value1> <data value2> ..<data valueN>
sw/sh/sb <-k> <virtual address in hex> <data value1> <data value2> ..<data valueN>
fw/fh/fb <physical address in hex> <data value> <length>
fw/fh/fb <-k> <virtual address in hex> <data value> <length>
-s (currently works with physical addresses for d*/s*/f* commands
and virtual addresses for s*/f* commands)



simo@RT-AX86U-5750:/tmp/home/root#


Running /jffs/addons/wireguard/wg_server from console works good.

You have an incompatible router/firmware or is potentially corrupt?

Do you get the same responses as shown below??

which sh

/opt/bin/sh

sh --help

BusyBox v1.25.1 (2022-03-25 10:23:25 EDT) multi-call binary.

Usage: sh [-/+OPTIONS] [-/+o OPT]... [-c 'SCRIPT' [ARG0 [ARGS]] / FILE [ARGS]]

Unix shell interpreter

uname  -a

Linux RT-AX86U-22B0 4.1.52 #2 SMP PREEMPT Fri Mar 25 11:09:29 EDT 2022 aarch64 ASUSWRT-Merlin

 

[simowdas]

The only diffrent is location of sh. Minę is on /bin/sh

 

[Martineau]

The only diffrent is location of sh. Minę is on /bin/sh

It appears that memaccess is being executed rather than sh

memaccess

*** Usage:
dw/dh/db <physical address in hex> <number>
dw/dh/db <-k> <virtual address in hex> <number>
sw/sh/sb <physical address in hex> <data value1> <data value2> ..<data valueN>
sw/sh/sb <-k> <virtual address in hex> <data value1> <data value2> ..<data valueN>
fw/fh/fb  <physical address in hex> <data value> <length>
fw/fh/fb <-k> <virtual address in hex> <data value> <length>
  -s (currently works with physical addresses for d*/s*/f* commands
and virtual addresses for s*/f* commands)

set | grep PATH

PATH='/opt/bin:/opt/sbin:/bin:/usr/bin:/sbin:/usr/sbin'

Symlinks.....

ls -lahF /usr/sbin | grep memaccess

Old bug....

vpnclient1 up / down scripts (openvpn) ac86u help needed

See two error lines. May 12 10:48:03 openvpn-event[31845_***DEBUG]: Error: Address /jffs/scripts/vpnclient1-route-pre-down is invalid May 12 10:48:17 openvpn-event[32176_***DEBUG]: Error: Address /jffs/scripts/vpnclient1-route-up is invalid Here are those two again. vpnclient1-route-pre-down...

www.snbforums.com

 

[simowdas]

ASUSWRT-Merlin RT-AX86U 386.5_2 Fri Mar 25 14:23:26 UTC 2022
simo@RT-AX86U-5750:/tmp/home/root# ls -lahF /usr/sbin | grep memaccess
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 db -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 dh -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 dw -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 fb -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 fh -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 fw -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 sb -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 sh -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 sw -> /bin/memaccess*
simo@RT-AX86U-5750:/tmp/home/root#


So the question is how to fix it? I can change symlink, but after firmware upgrade it can go back.

 

[Martineau]

ASUSWRT-Merlin RT-AX86U 386.5_2 Fri Mar 25 14:23:26 UTC 2022
simo@RT-AX86U-5750:/tmp/home/root# ls -lahF /usr/sbin | grep memaccess
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 db -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 dh -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 dw -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 fb -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 fh -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 fw -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 sb -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 sh -> /bin/memaccess*
lrwxrwxrwx 1 simo root 14 Mar 25 16:11 sw -> /bin/memaccess*
simo@RT-AX86U-5750:/tmp/home/root#


So the question is how to fix it? I can change symlink, but after firmware upgrade it can go back.

So what is your PATH= ?

 

[simowdas]

LD_LIBRARY_PATH='/lib:/usr/lib:/lib/aarch64'
PATH='/bin:/usr/bin:/sbin:/usr/sbin:/home/simo:/mmc/sbin:/mmc/bin:/mmc/usr/sbin:/mmc/usr/bin:/opt/sbin:/opt/bin:/opt/usr/sbin:/opt/usr/bin'
simo@RT-AX86U-5750:/tmp/home/root#

 

[Martineau]

LD_LIBRARY_PATH='/lib:/usr/lib:/lib/aarch64'
PATH='/bin:/usr/bin:/sbin:/usr/sbin:/home/simo:/mmc/sbin:/mmc/bin:/mmc/usr/sbin:/mmc/usr/bin:/opt/sbin:/opt/bin:/opt/usr/sbin:/opt/usr/bin'
simo@RT-AX86U-5750:/tmp/home/root#

As you are apparently already successfully customising the path (/home/simo:) then ensure that the correct Unix Shell Interpreter i.e. Busybox sh is found/executed first i.e. before the rogue Symlink to memaccess

(Or I believe someone did a reinstall and it automagically resolved itself.)

 

[ZebMcKayhan]

@archiel whenever you get your ipset restored properly at boot (hooping a sleep 10 at top of nat-start does the trick, I need this to get predictable outcome), could you try to add the ipset in wgm:

E:Option ==> stop wg11
E:Option ==> peer wg11 add ipset wg11-mac
E:Option ==> peer wg11 upd ipset wg11-mac dstsrc src
E:Option ==> start wg11

Before doing this, comment the following in your wg11-up.sh:

ip rule add from all fwmark 0x1000 table 121 prio 9900 
ip -6 rule add from all fwmark 0x1000 table 121 prio 9900 
echo 2 > /proc/sys/net/ipv4/conf/wg11/rp_filter
iptables -t mangle -I PREROUTING -m set --match-set wg11-mac src -j MARK --set-mark 0x1000/0x1000 
ip6tables -t mangle -I PREROUTING -m set --match-set wg11-mac src -j MARK --set-mark 0x1000/0x1000

as wgm applies these for you. Then please check so that wgm sets everything up and if not please report what's missing. Meanwhile you could enable the missing pieces to get it working.

 

[simowdas]

As you are apparently already successfully customising the path (/home/simo:) then ensure that the correct Unix Shell Interpreter i.e. Busybox sh is found/executed first i.e. before the rogue Symlink to memaccess

(Or I believe someone did a reinstall and it automagically resolved itself.)

I don't understand. The first on PATH is /bin where correct sh is.

 

[Martineau]

I don't understand. The first on PATH is /bin where correct sh is.

I've uploaded wireguard_manager Beta v4.16bB

I've now removed the method of calling external scripts i.e. 'sh {INSTALL_DIR}wireguard_script_name' so executing the '/usr/sbin/sh' symlink to execute memaccess rather than the Shell Interpreter should no longer occur.

To test

e  = Exit Script [?]

E:Option ==> uf dev

    Router RT-AX86U Firmware (v386.5_2)

    [✔] Entware Architecture arch=aarch64

<snip>

 

[Martineau]

add the ipset in wgm:

E:Option ==> peer wg11 add ipset wg11-mac

FYI wireguard_manager Beta v4.16bB will now automatically FORCE dstsrc src (dstsrc dst is obviously invalid) if the IPSet is Type hash:mac

Verify using

E:Option ==> peer wg11

So the following is no longer explicitly required.

E:Option ==> peer wg11 upd ipset wg11-mac dstsrc src

NOTE: By default ALL IPSets will have tag dstsrc dst applied when they are bound to a 'client' Peer, so the 'dimension' tag(s) should be reviewed/checked.

i.e. If you have a two-dimension MAC IPSet such as Type: hash:ip,mac or Type: bitmap:ip,mac then you will need to manually explicitly tag the IP as being either src or dst with src MAC

e.g.

peer wg11 upd ipset wg11-mac dstsrc src,src

or

peer wg11 upd ipset wg11-mac dstsrc dst,src

 

[simowdas]

I've uploaded wireguard_manager Beta v4.16bB

I've now removed the method of calling external scripts i.e. 'sh {INSTALL_DIR}wireguard_script_name' so executing the '/usr/sbin/sh' symlink to execute memaccess rather than the Shell Interpreter should no longer occur.

To test

e  = Exit Script [?]

E:Option ==> uf dev

    Router RT-AX86U Firmware (v386.5_2)

    [✔] Entware Architecture arch=aarch64

<snip>

And now everythink works perfect Thanks for help

 

[Martineau]

I remember something about wgm scanning the set for INET or INET6 to determine if the rules should be in iptables or ip6tables. this would go in both, hence my comment that wgm might need to recognize the hash:mac type of set and put in firewall rules for both (or as appropriate depending on router setup). or maybee it already does?

FYI,

wireguard_manager Beta v4.16bB now makes an exception for IPSets of Type: hash:mac whereby, rather than insisting on creating a duplicate IPv6 IPset (mirroring the contents of the IPv4 MAC IPSet), the IPv6 firewall rule will now also be created if appropriate.

 

[ZebMcKayhan]

FYI,

wireguard_manager Beta v4.16bB now makes an exception for IPSets of Type: hash:mac whereby, rather than insisting on creating a duplicate IPv6 IPset (mirroring the contents of the IPv4 MAC IPSet), the IPv6 firewall rule will now also be created if appropriate.

Great! Now hooping @archiel testing turns out ok, then maybee this is an alternative for policy based routing for ipv6, especially when having dynamic ip and/or using SLAAC privacy extension.

Wrote something short about creating, store and restore these ipsets:
https://github.com/ZebMcKayhan/WireguardManager/blob/main/README.md#create-and-setup-ipsets

But all is not fully tested yet.