x3mrouting liststats ipset becoming 0 after reboot

[ComputerSteve]

I keep having a problem on certain reboots my liststats are erased. Is anyone having that on the latest merlin 386.5? I have to rerun the commands to recreate them.

 

[chongnt]

How to you create your ipset? When you run “cru l | grep ipset”, do you see cronjob that save your ipset?
You should see something like this. It is stored in /opt/tmp directory.

0 2 * * * ipset save $IPSET_NAME > /opt/tmp/$IPSET_NAME

 

[ComputerSteve]

How to you create your ipset? When you run “cru l | grep ipset”, do you see cronjob that save your ipset?
You should see something like this. It is stored in /opt/tmp directory.

0 2 * * * ipset save $IPSET_NAME > /opt/tmp/$IPSET_NAME

I created it with this command exactly: sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 Cablevision-ASN asnum=AS6128 # Route-Altice-One traffic via Wan

 

[ComputerSteve]

How to you create your ipset? When you run “cru l | grep ipset”, do you see cronjob that save your ipset?
You should see something like this. It is stored in /opt/tmp directory.

0 2 * * * ipset save $IPSET_NAME > /opt/tmp/$IPSET_NAME

When I run cru l | grep ipset it doesn't appear to do anything with SSH. I mean it works for some time but then it seems like they get deleted on there own. Like I just reran the commands to make the IPSETs and now it shows this
AMAZONPRIME - 6397
Cablevision-ASN - 68
HULU - 3954

However.. Sometimes I reboot and it shows 0 and then the rules don't work.

 

[chongnt]

When I run cru l | grep ipset it doesn't appear to do anything with SSH. I mean it works for some time but then it seems like they get deleted on there own. Like I just reran the commands to make the IPSETs and now it shows this
AMAZONPRIME - 6397
Cablevision-ASN - 68
HULU - 3954

However.. Sometimes I reboot and it shows 0 and then the rules don't work.

By right you should have cronjob to save ipset entries. You can try to manually save a copy first.

here is how my output looks like

admin@RT-AC86U-DBA8:/tmp/home/root# cru l | grep ipset
0 2 * * * ipset save Astro > /opt/tmp/Astro #Astro#
…snipped…
admin@RT-AC86U-DBA8:/tmp/home/root# wc -l /opt/tmp/Astro 
765 /opt/tmp/Astro
admin@RT-AC86U-DBA8:/tmp/home/root# liststats
Astro - 764

 

[ComputerSteve]

Here's what happens. When I run cru l I don't see anything mentioned about IPSET's

 

[chongnt]

Here's what happens. When I run cru l I don't see anything mentioned about IPSET's

Hmm, not sure what went wrong. You mentioned you have to recreate the rules after reboot. By right you don’t need to. The rules are stored in /jffs/scripts/nat-start and /jffs/config/dnsmasq.conf.add
Can you check if the rules exists in these files?

 

[ComputerSteve]

Hmm, not sure what went wrong. You mentioned you have to recreate the rules after reboot. By right you don’t need to. The rules are stored in /jffs/scripts/nat-start and /jffs/config/dnsmasq.conf.add
Can you check if the rules exists in these files?

How do I do that?

 

[ComputerSteve]

Also I tried this command x3mRouting ipset_name=AMAZONPRIME asnum=AS16509

Admin@AX11000-DEN-418:/tmp/home/root# x3mRouting ipset_name=AMAZONPRIME asnum=AS

(x3mRouting): 10876 Starting Script Execution ipset_name=AMAZONPRIME asnum=AS16509
(x3mRouting): 10876 sh /jffs/scripts/x3mRouting/x3mRouting.sh ipset_name=AMAZONPRIME asnum=AS16509 added to /jffs/scripts/nat-start
(x3mRouting): 10876 Completed Script Execution

 

[ComputerSteve]

So this is my nat-start:
#!/bin/sh
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZONPRIME asnum=AS16509
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 HULU asnum=AS20940
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 Cablevision-ASN asnum=AS6128
sh /jffs/scripts/x3mRouting/x3mRouting.sh ipset_name=AMAZONPRIME asnum=AS16509
sh /jffs/scripts/x3mRouting/x3mRouting.sh ipset_name=HULU asnum=AS20940
sh /jffs/scripts/x3mRouting/x3mRouting.sh ipset_name=Cablevision-ASN asnum=AS6128

I DO NOT SEE A /jffs/config/dnsmasq.conf.add when I browse using SFTP cyperduck..

 

[chongnt]

So this is my nat-start:
#!/bin/sh
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZONPRIME asnum=AS16509
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 HULU asnum=AS20940
sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 Cablevision-ASN asnum=AS6128
sh /jffs/scripts/x3mRouting/x3mRouting.sh ipset_name=AMAZONPRIME asnum=AS16509
sh /jffs/scripts/x3mRouting/x3mRouting.sh ipset_name=HULU asnum=AS20940
sh /jffs/scripts/x3mRouting/x3mRouting.sh ipset_name=Cablevision-ASN asnum=AS6128

I DO NOT SEE A /jffs/config/dnsmasq.conf.add when I browse using SFTP cyperduck..

I think you are right that you don't see this in dnsmasq.conf.add because you are using asnum. My use case is a bit different as I don't use asnum. Instead I use dnsmasq.
I just try it. Looks like ipset save cronjob is created when using dnsmasq but not asnum. Probably that is the expected behavior. Hope someone that use asnum can share their experience.

admin@RT-AC86U-DBA8:/# sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZONPRIME asnum=AS16509
(x3mRouting.sh): 29579 Starting Script Execution 1 0 AMAZONPRIME asnum=AS16509
(x3mRouting.sh): 29579 IPSET created: AMAZONPRIME hash:net family inet hashsize 1024 maxelem 65536

(x3mRouting.sh): 29579 Selective Routing Rule via WAN created for AMAZONPRIME fwmark 0x8000/0x8000
(x3mRouting.sh): 29579 iptables -t mangle -D PREROUTING -i br0 -m set --match-set AMAZONPRIME dst -j MARK --set-mark 0x8000/0x8000 2>/dev/null added to /jffs/scripts/x3mRouting/vpnclient1-route-up
(x3mRouting.sh): 29579 iptables -t mangle -A PREROUTING -i br0 -m set --match-set AMAZONPRIME dst -j MARK --set-mark 0x8000/0x8000 added to /jffs/scripts/x3mRouting/vpnclient1-route-up
(x3mRouting.sh): 29579 iptables -t mangle -D PREROUTING -i br0 -m set --match-set AMAZONPRIME dst -j MARK --set-mark 0x8000/0x8000 2>/dev/null added to /jffs/scripts/x3mRouting/vpnclient1-route-pre-down
(x3mRouting.sh): 29579 sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0 AMAZONPRIME asnum=AS16509 added to /jffs/scripts/nat-start
(x3mRouting.sh): 29579 Completed Script Execution

admin@RT-AC86U-DBA8:/# sh /jffs/scripts/x3mRouting/x3mRouting.sh 1 0  ipset_name=test dnsmasq=test.com
(x3mRouting.sh): 1089 Starting Script Execution 1 0 ipset_name=test dnsmasq=test.com
(x3mRouting.sh): 1089 CRON schedule created: #test# '0 2 * * * ipset save test'
(x3mRouting.sh): 1089 Completed Script Execution

 

[ComputerSteve]

So I think I found the problem. I have inactive rules in VPN Director. I notice that if I enable or disable or even modify those rules then after I reboot the router the IPSET lists go back to 0. It seems to be an incompatibility with x3mrouting and VPN Director. I now seem to have fixed it by just having the four devices I actually use with VPN Director. So I removed all the rules that I have created just in case I needed them and now it seems stable. It seems like editing the VPN Director rule list it messes up the IPSETS.

 

[chongnt]

So I think I found the problem. I have inactive rules in VPN Director. I notice that if I enable or disable or even modify those rules then after I reboot the router the IPSET lists go back to 0. It seems to be an incompatibility with x3mrouting and VPN Director. I now seem to have fixed it by just having the four devices I actually use with VPN Director. So I removed all the rules that I have created just in case I needed them and now it seems stable. It seems like editing the VPN Director rule list it messes up the IPSETS.

Great that you found the issue. Just don't install x3mRouting option 2. Option 2 OVPN Client screen does not work with VPN Director. Use option 3.

 

[ComputerSteve]

Great that you found the issue. Just don't install x3mRouting option 2. Option 2 OVPN Client screen does not work with VPN Director. Use option 3.

I am. That is why I thought I wouldn't have any issues. It appears if you play around with the rules list and your using option 3 it messes up the IPSET lists after a reboot. Must be some incompatibility but at least it works if I don't touch anything. Hopefully someone takes over x3mrouting to update it for Director.

 

[ComputerSteve]

Again randomly all the IPSET lists turned to 0 Why does this keep happening? is anyone else experiencing this ?
I do notice this in the logs (x3mvpnrouting.sh): 32112 00 Deleting fwmark 0x1000/0x1000

 

[ComputerSteve]

Any idea on why every reboot my ipsets become 0 ? I have never had this problem before @RMerlin

 

[dave14305]

Any idea on why every reboot my ipsets become 0 ? I have never had this problem before @RMerlin

The script uses https://api.bgpview.io to download the ipset content from. This site has been known to ratelimit quite a bit (it has affected Skynet as well). You’ll have to look deeper into what‘s happening when the script tries to download the asn data. The script author is MIA for a long time.

 

[ComputerSteve]

So is there a way to make IPSets without this script and also it works it just like erases after reboot.

 

[ZebMcKayhan]

So is there a way to make IPSets without this script and also it works it just like erases after reboot.

Certainly, most likely. The question is if you or anyone else would invest the time to do it.

If you search x3mRouting.sh for Download_ASN_Ipset_List() you will find how the asn ipsets are downloaded and put in the set.

But populating the ipsets are only part of the story. You also need firewall rules to mark packages with destination ip matching the ipset with a unique mark. You also need to setup routing to route marked packages according to your needs. And disable reverse path filtering on choosen interface...

 

[ComputerSteve]

So I just want to say I have downgraded my firmware to 386.2_6 I now do not have that problem with the GT-AX11000. I am wondering is x3mrouting working for others on the new firmware ? I know your supposed to use the no GUI option which I have been doing. The rules get created but after a few reboots they delete and become 0. I also notice that if I modified any vpn director rules it also caused the IPSETS to get erased.