What's new

firewall-start being executed twice by IPDfw (what is it?)

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

bengalih

Senior Member
So this question has been posted a couple of times with varying degrees of answers, such as:

The general consensus by @RMerlin is that the code only calls this to run once.
Looking at my logs below it would appear that "IDPfw" (/dev/idpfw) is causing the firewall-start script to be called again.
I can only guess this stands for "Intrusion Detection Prevention Firewall", but I have no idea what service that actually is.
Is it something I can disable? If not, then it would appear that normal operation does require the firewall-start script being executed twice.

The first time it executes is about 2:24 seconds after router boot, which appears to be the "normal" firewall-start call. You can see it below at 21:05:14:

Code:
May  7 21:05:14 kernel: nf_conntrack_rtsp v0.6.21 loading
May  7 21:05:14 kernel: nf_nat_rtsp v0.6.21 loading
May  7 21:05:14 custom_script: Running /jffs/scripts/firewall-start (args: eth0)

The second time it executes is about a minute later - you can see it at 21:06:11 below:

Code:
May  7 21:06:04 kernel: Init chrdev /dev/idpfw with major 191
May  7 21:06:05 kernel: IDPfw: IDPfw is ready
May  7 21:06:05 kernel: sizeof forward pkt param = 192
May  7 21:06:05 BWDPI: fun bitmap = 3
May  7 21:06:11 custom_script: Running /jffs/scripts/firewall-start (args: eth0)
 
So this question has been posted a couple of times with varying degrees of answers, such as:

The general consensus by @RMerlin is that the code only calls this to run once.
Looking at my logs below it would appear that "IDPfw" (/dev/idpfw) is causing the firewall-start script to be called again.
I can only guess this stands for "Intrusion Detection Prevention Firewall", but I have no idea what service that actually is.
Is it something I can disable? If not, then it would appear that normal operation does require the firewall-start script being executed twice.

The first time it executes is about 2:24 seconds after router boot, which appears to be the "normal" firewall-start call. You can see it below at 21:05:14:

Code:
May  7 21:05:14 kernel: nf_conntrack_rtsp v0.6.21 loading
May  7 21:05:14 kernel: nf_nat_rtsp v0.6.21 loading
May  7 21:05:14 custom_script: Running /jffs/scripts/firewall-start (args: eth0)

The second time it executes is about a minute later - you can see it at 21:06:11 below:

Code:
May  7 21:06:04 kernel: Init chrdev /dev/idpfw with major 191
May  7 21:06:05 kernel: IDPfw: IDPfw is ready
May  7 21:06:05 kernel: sizeof forward pkt param = 192
May  7 21:06:05 BWDPI: fun bitmap = 3
May  7 21:06:11 custom_script: Running /jffs/scripts/firewall-start (args: eth0)
Might be trendmicro related, but I have no clue. Just a blind guess. Do you have AI Protect enabled?
 
No I don't. I pretty much have all those extraneous features disabled.
Curiously enough I did also notice this in my process output:

Code:
root@router-asus:/jffs/scripts# ps | grep wred
 3041 root  7248 S    wred -B
 3044 root  7248 S    wred -B
 3045 root  7248 S    wred -B
 3067 root  7644 S    bwdpi_wred_alive
 3223 root  7248 S    wred -B
 3224 root  7248 S    wred -B
 3225 root  7248 S    wred -B
 3226 root  7248 S    wred -B
 3227 root  7248 S    wred -B
 3228 root  7248 S    wred -B
 3229 root  7248 S    wred -B
 3230 root  7248 S    wred -B
 8225 root  1456 S    grep wred

Google search indicates this wred is part of TrendMicro DPI.
Again, all my TrendMicro stuff appears disabled....so I don't know why any of these are running.
 
Are you using QoS?

I don't see those messages on startup and my firewall is only started once. However that would not be the case if for example I was running a VPN server. So different setups produce different results.

I don't have any of the TM stuff enabled. Go to Administration > Privacy and Withdraw if there's an option to do so. You will still see however, IDPfw messages and firewall restarts when navigating to the Adapative QoS > Bandwidth Monitor page in the GUI.
 
No I don't. I pretty much have all those extraneous features disabled.
Curiously enough I did also notice this in my process output:

Code:
root@router-asus:/jffs/scripts# ps | grep wred
 3041 root  7248 S    wred -B
 3044 root  7248 S    wred -B
 3045 root  7248 S    wred -B
 3067 root  7644 S    bwdpi_wred_alive
 3223 root  7248 S    wred -B
 3224 root  7248 S    wred -B
 3225 root  7248 S    wred -B
 3226 root  7248 S    wred -B
 3227 root  7248 S    wred -B
 3228 root  7248 S    wred -B
 3229 root  7248 S    wred -B
 3230 root  7248 S    wred -B
 8225 root  1456 S    grep wred

Google search indicates this wred is part of TrendMicro DPI.
Again, all my TrendMicro stuff appears disabled....so I don't know why any of these are running.
To add to what @ColinTaylor has said, you will need to disable any conglomarate softwares that share the same trendmicro agreement disclaimer under the privacy section. This includes QoS, AI Protect, DDNS services, parental controls, and any firewall extras such as Open Nat or WTFast.
 
So this question has been posted a couple of times with varying degrees of answers, such as:

The general consensus by @RMerlin is that the code only calls this to run once.
Looking at my logs below it would appear that "IDPfw" (/dev/idpfw) is causing the firewall-start script to be called again.
I can only guess this stands for "Intrusion Detection Prevention Firewall", but I have no idea what service that actually is.
Is it something I can disable? If not, then it would appear that normal operation does require the firewall-start script being executed twice.

The first time it executes is about 2:24 seconds after router boot, which appears to be the "normal" firewall-start call. You can see it below at 21:05:14:

Code:
May  7 21:05:14 kernel: nf_conntrack_rtsp v0.6.21 loading
May  7 21:05:14 kernel: nf_nat_rtsp v0.6.21 loading
May  7 21:05:14 custom_script: Running /jffs/scripts/firewall-start (args: eth0)

The second time it executes is about a minute later - you can see it at 21:06:11 below:

Code:
May  7 21:06:04 kernel: Init chrdev /dev/idpfw with major 191
May  7 21:06:05 kernel: IDPfw: IDPfw is ready
May  7 21:06:05 kernel: sizeof forward pkt param = 192
May  7 21:06:05 BWDPI: fun bitmap = 3
May  7 21:06:11 custom_script: Running /jffs/scripts/firewall-start (args: eth0)
And upon inspection of some older logs, a IDPfw plays a role as the trendmicro forwarding module. At least it did, I don't know if it still does.


Mar 23 10:53:41 kernel: IDPfw: TrendMicro forward module ver-1.0.34

The firewall-start(args: eth0) should happen just as IDPfw loads the modulation part for eth0 and udhcpc might even start here as well.
 
Last edited:
Are you using QoS?

I don't see those messages on startup and my firewall is only started once. However that would not be the case if for example I was running a VPN server. So different setups produce different results.

I don't have any of the TM stuff enabled. Go to Administration > Privacy and Withdraw if there's an option to do so. You will still see however, IDPfw messages and firewall restarts when navigating to the Adapative QoS > Bandwidth Monitor page in the GUI.
Oh man. I don't know whether to hug you or punch you in the face...LOL.
Really, you started me down a path which looks like it has a happy ending now, but it was a bumpy road.

So, I went ahead and withdrew from the Privacy settings. IMMEDIATELY on doing so I saw my free RAM usage jump from about 80-90MB to about 150+. I mean doing this effectively halved my RAM utilization!
Everything still seemed to work ok, but on a reboot I couldn't connect to WAN.
It took me about an hour or so debugging and researching to figure out why. Basically here is the deal:

I have a custom config here where I run wpa_supplicant to do 802.11x auth to AT&T fiber so I can bypass using their modem. This worked great when my WAN interface was eth0. However, whatever happens in the background when you do that final "withdraw" from the EULA and effectively kills all AI services also seems to reconfigure the whole networking stack so that the WAN interface becomes a VLAN (vlan2 in my case).

I did a bit of research (still ongoing) and apparently this is a known issue as AT&T expects the packets to be tagged. When it was on eth0, something in the router was doing the tagging, but on the vlan they aren't tagged. I'm still looking into this - because I don't think tagging vlan traffic is possible on this router(?) So at that point my choices were keep the free RAM or put AT&T's equipment back in to do authentication. I didn't like either of those options so I dug a bit more.

I found that when choosing the "Withdraw" option the following things happened on the router:
1) nvram sets the following:
bwdpi_db_enable=0
TM_EULA=0

2) I saw the following executed in syslog:
Code:
notify_rc restart_wrs;restart_qos;restart_firewall

So playing around with the above I found out that the IPDfw was dependent on qos.
The trick is if bwdpi_db_enable=0 on boot up then my system gets configured to use vlan2 for WAN.
So what I did is write some scripts to run on boot to toggle this to 0 and then restart qos.
After qos restarts I set this back to 1 so that on the next boot my system net configuration will stay how I need it.
I've done several reboots now and this seems stable.
The only downside I am seeing so far (not so much a downside, but an inefficiency) is that I have to allow it to start up with bwdpi_db_enable=0 so my interfaces are configured properly. Then I switch it off. So for a few seconds during boot it eats up the memory and takes a few seconds more to process. A good trade-off I think.

I'm not using QOS, and if I was I'm not sure if this would break it. QOS *is* still running as a service, just not with the IPDfw component. I think if you wanted to use QOS and this broke it, the extra RAM usage is just the trade-off you get for that feature.

Some more observations I see from simply toggling the bwdpi_db_enable bit and restarting qos.:

When bwdpi_db_enable=1 (TM/AI prot or whatever is on), the following is in my ps output:

Code:
when enabled:
 1042 root  7644 S    bwdpi_check
14821 root  7644 S    bwdpi_wred_alive
14831 root  3472 S    dcd -i 3600 -p 43200 -b -d /tmp/bwdpi/
14832 root  3472 S    dcd -i 3600 -p 43200 -b -d /tmp/bwdpi/
14833 root  3472 S    dcd -i 3600 -p 43200 -b -d /tmp/bwdpi/
14834 root  3472 S    dcd -i 3600 -p 43200 -b -d /tmp/bwdpi/
14835 root  3472 S    dcd -i 3600 -p 43200 -b -d /tmp/bwdpi/
14836 root  3472 S    dcd -i 3600 -p 43200 -b -d /tmp/bwdpi/
14857 root  3472 S    dcd -i 3600 -p 43200 -b -d /tmp/bwdpi/


14841 root  7272 S    wred -B
14842 root  7272 S    wred -B
14843 root  7272 S    wred -B
14867 root  7272 S    wred -B
14868 root  7272 S    wred -B
14869 root  7272 S    wred -B
14870 root  7272 S    wred -B
14871 root  7272 S    wred -B
14872 root  7272 S    wred -B
14873 root  7272 S    wred -B
14874 root  7272 S    wred -B

When bwdpi_db_enable=0 (TM/AI prot or whatever is OFF), the only process from above that is running is bwdpi_check

So right now, everything seems working great for my needs.
I am a bit suspicious that this hack *may* have also broken Traffic Analyzer Statistics which I do like to use (although not as necessary since I don't have a cap with current ISP). Hard to say yet with all the up and down I've had...I will come back and definitively state one way or another in the next day or so.

Oh..and I forgot to mention: On the second or third reboot when my WAN was broken, somehow my flash drive got corrupted (instead of ext4 it was being recognized as NTFS, but obviously unreadable). Luckily I had an image backup and I was able to get back to my current working form in just an hour or two, but it was touch and go there for a bit - which is partly why I said above it was such a bumpy road.

Anyway, I'm interested in your and @SomeWhereOverTheRainBow 's thoughts based on what you theorized and what I have done.
At the very least hopefully this could be helpful to someone else. At the very least knowing that simply opting out of the Privacy EULA frees up so much memory even when you already had all the other services disabled!
 
The eth0/vlan2 differences on the RT-AC68U are caused by hardware acceleration. The WAN interface is vlan2 when you have hardware acceleration enabled, otherwise it's eth0.

Anything that uses DPI has to turn off hardware acceleration, that includes "Traffic Analyzer - Statistic". If you don't need the higher internet throughput provided by hardware acceleration you could just disable it in the "LAN - Switch Control" settings.
 
Last edited:
The eth0/vlan2 differences on the RT-AC68U are caused by hardware acceleration. The WAN interface is vlan2 when you have hardware acceleration enabled, otherwise it's eth0.

Anything that uses DPI has to turn off hardware acceleration, that includes "Traffic Analyzer - Statistic". If you don't need the higher internet throughput provided by hardware acceleration you could just disable it in the "LAN - Switch Control" settings.

So right now (as was before) my "NAT Acceleration" is set to auto and it states next to it (in yellow) "CTF (Cut Through Forwarding) is enabled."

If I set it to "disable" (the only other option), my hard-wired computers get only about 300Mbps on speed tests as opposed to the ~850Mbps they can achieve over my fiber when it is set to "auto."

Additionally, "Traffic Analyzer - Statistic" has appeared to work without issue for me in this configuration for years.

With the changes I made above, all of this still holds true. The setting is enabled, I get the higher d/l speeds and as of right now it also appears that the ""Traffic Analyzer - Statistic" is still updating (though I want to give it another few hours of data).

Can you comment on this, because from the way I read what you wrote this would seem to not be possible? Maybe I am just confused?

Oh...also to note, I know that my WAN still works whether or not I enable or disable the Acceleration because it was actually disabled initially which is why I was having slower internet speeds. I had to enable it, but in both cases my WAN was still on eth0 (or else it wouldn't have worked).
 
With the changes I made above, all of this still holds true. The setting is enabled, I get the higher d/l speeds and as of right now it also appears that the ""Traffic Analyzer - Statistic" is still updating (though I want to give it another few hours of data).

Can you comment on this, because from the way I read what you wrote this would seem to not be possible? Maybe I am just confused?
No you're not confused. My memory was incorrect, I haven't used my RT-AC68U for a year.

Thinking back, the big advantage of "Adaptive QoS" (which uses DPI) when introduced was that it didn't slow down the throughput in the same way that completely disabling hardware acceleration did. The later was required to get Traditional QoS to work correctly. So you're correct when you say Traffic Analyzer still works with CTF enabled. However, if you have withdrawn from the TM licence and then enable Traffic Analyzer I think you then have to agree to the licence again.

Oh...also to note, I know that my WAN still works whether or not I enable or disable the Acceleration because it was actually disabled initially which is why I was having slower internet speeds. I had to enable it, but in both cases my WAN was still on eth0 (or else it wouldn't have worked).
Sorry, I can't really follow your setup. But even when vlan2 is configured as the WAN interface the eth0 interface still exists, it just doesn't have an IP address configured on it. So maybe that's relevant to what you doing with your ISP. Usually in scripts we use nvram get wan0_ifname to obtain the current primary WAN interface name.
 
Sorry, I can't really follow your setup. But even when vlan2 is configured as the WAN interface the eth0 interface still exists, it just doesn't have an IP address configured on it. So maybe that's relevant to what you doing with your ISP. Usually in scripts we use nvram get wan0_ifname to obtain the current primary WAN interface name.

Basically, ever since I setup my AT&T connection my `nvram get wan0_ifname` has been eth0 (and was before that).
I don't know the specifics, but the people that got this AT&T bypass setup working stated that the AI Protection (or something related) would tag all traffic to the WAN as VLAN. This works when `nvram get wan0_ifname= eth0`.

However, if `nvram get wan0_ifname= vlan2` (which it is if I let the router reboot normally after withdrawing from Privacy with all other AI protection off) then apparently whatever on the router normally tagged this traffic no longer does and it is rejected by the upstream ISP. If I could find a way to tag vlan2 traffic that would be a way around it. Best I can tell though is no one knows how to do it on this router.

Anyway - my point is that if `nvram get wan0_ifname= eth0` I get Internet. If `nvram get wan0_ifname= anything else` I don't. I apparently need the WAN IP bound to eth0 to get tagged.

I have never had any issue getting Internet. Whether or not AI was disabled or enabled, if Traffic monitor was on or off, or if Acceleration was on or off. I can test again, but I'm 99.9% sure that means I was always `wan0_ifname= eth0`.

Sorry if that's not what you meant, I'm just trying to clear up my side. If this still sounds weird to you let me know and I can do a test to confirm.
 
Ok - so update on the "Traffic Analyzer - Statistics". It definitely appears that bwdpi_db_enable is directly tied to this feature:


Now @RMerlin states there that disabling it "merely enables/disable apps analysis. The global traffic monitoring itself can't be enabled/disabled."

But, from what I am seeing, since disabling this and restarting QoS, I haven't gotten any updates to my Daily Traffic, whether viewing "Device" or "Apps". Maybe I misunderstand what he is saying, but it appears that if you disable this you will lose all "Traffic Analyzer - Statistics."

I do think for now I am going to live with it off, if only to test that the rest of my setup is otherwise stable in this configuration.

The WebUI will show "Traffic Analyzer - Statistics" on or off dependent on the nvram variable solely. So even if it is set to 0 but you haven't restarted QoS yet you are probably still capturing data (you just won't be able to see it until you toggle it back on).

It seems that ASUS didn't put very good logic/checks into these services since, as was mentioned (and what apparently I did), by using Traffic Analyzer but never option out to withdraw.

Anyway I still think the most interesting part of this is that the WAN interface apparently makes the decision to bind to eth0 vs. vlan2 based on whether "bwdpi_db_enable" is enabled or not at boot time (though there may also be some other AI variables that cause this), and that I can disable all the services after booting up in enabled mode to force eth0.
 
Ok - so update on the "Traffic Analyzer - Statistics". It definitely appears that bwdpi_db_enable is directly tied to this feature:


Now @RMerlin states there that disabling it "merely enables/disable apps analysis. The global traffic monitoring itself can't be enabled/disabled."

But, from what I am seeing, since disabling this and restarting QoS, I haven't gotten any updates to my Daily Traffic, whether viewing "Device" or "Apps". Maybe I misunderstand what he is saying, but it appears that if you disable this you will lose all "Traffic Analyzer - Statistics."

I do think for now I am going to live with it off, if only to test that the rest of my setup is otherwise stable in this configuration.

The WebUI will show "Traffic Analyzer - Statistics" on or off dependent on the nvram variable solely. So even if it is set to 0 but you haven't restarted QoS yet you are probably still capturing data (you just won't be able to see it until you toggle it back on).

It seems that ASUS didn't put very good logic/checks into these services since, as was mentioned (and what apparently I did), by using Traffic Analyzer but never option out to withdraw.

Anyway I still think the most interesting part of this is that the WAN interface apparently makes the decision to bind to eth0 vs. vlan2 based on whether "bwdpi_db_enable" is enabled or not at boot time (though there may also be some other AI variables that cause this), and that I can disable all the services after booting up in enabled mode to force eth0.
Well I think you have achieved the better understanding you were seeking in regards to the original purpose of this post. In light of everything, I have always added a delete version to each of my firewall-start rules that runs before the main rule gets ran.

alternatively you can run iptables -c and rely on the return value to determine if the firewall-start script appends the rules. I really see no actual difference. This would eliminate the possible production of duplicate rules if the firewall-start gets ran and the firewall is not actually cleared. Which appears to be what happens when trend micro runs.

Alternatively, here is a one liner that clears out duplicate rules. I put this at the end of my firewall-start script.

Code:
iptables-save | awk '/^COMMIT$/ { delete x; }; !x[$0]++' | iptables-restore; ip6tables-save | awk '/^COMMIT$/ { delete x; }; !x[$0]++' | ip6tables-restore
 
Well I think you have achieved the better understanding you were seeking in regards to the original purpose of this post. In light of everything, I have always added a delete version to each of my firewall-start rules that runs before the main rule gets ran.

alternatively you can run iptables -c and rely on the return value to determine if the firewall-start script appends the rules. I really see no actual difference. This would eliminate the possible production of duplicate rules if the firewall-start gets ran and the firewall is not actually cleared. Which appears to be what happens when trend micro runs.

Alternatively, here is a one liner that clears out duplicate rules. I put this at the end of my firewall-start script.

Code:
iptables-save | awk '/^COMMIT$/ { delete x; }; !x[$0]++' | iptables-restore; ip6tables-save | awk '/^COMMIT$/ { delete x; }; !x[$0]++' | ip6tables-restore
Thanks for the one liner, but no that wasn't my intent at all with this thread, nor the problem I was having.
All my firewall scripts issue delete commands before adding, so the firewall restarting multiple times was not an issue as far as that is concerned.

Initially, I was just interested *why* it was restarting twice because I saw nothing in the logs that would require it and I was not using any services that should have. Until I discovered the IDPfw as the clue as to what was requiring the restart. Once I found that I wanted to know what it was, and that let me down the rabbit hole discovering that if I was able to terminate it I was able to double my available RAM (after figuring out how to keep my WAN bound to eth0 as I discussed above).
 
Thanks for the one liner, but no that wasn't my intent at all with this thread, nor the problem I was having.
All my firewall scripts issue delete commands before adding, so the firewall restarting multiple times was not an issue as far as that is concerned.

Initially, I was just interested *why* it was restarting twice because I saw nothing in the logs that would require it and I was not using any services that should have. Until I discovered the IDPfw as the clue as to what was requiring the restart. Once I found that I wanted to know what it was, and that let me down the rabbit hole discovering that if I was able to terminate it I was able to double my available RAM (after figuring out how to keep my WAN bound to eth0 as I discussed above).
Consequently I also believe that is one of the firewall-start runs that creates duplicates, that is why I said the such, but I see that your quest was more for knowledge to achieve your desired outcome.
 
Consequently I also believe that is one of the firewall-start runs that creates duplicates, that is why I said the such, but I see that your quest was more for knowledge to achieve your desired outcome.
Yes you are correct that IDPfw causes the second restart. Technically, I don't think it is IDPfw but whatever is reading the nvram variable(s) I mention for TM that triggers the processes listed above (bwdpi, wred, etc). IDPfw is a consequence of whatever this other service is and when it tirggers the firewall script runs again.
When I started writing the thread I hadn't yet determined that IDPfw was the culprit, I just wanted to know why the second start and while writing the post I found that but continued to ask the question as it was sort of one and the same.

thanks for the input.
 

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top