What's new

MERLIN: RT-AC68U + ADBLOCKing...

  • SNBForums Code of Conduct

    SNBForums is a community for everyone, no matter what their level of experience.

    Please be tolerant and patient of others, especially newcomers. We are all here to share and learn!

    The rules are simple: Be patient, be nice, be helpful or be gone!

I've modified ryzhov_al's script to use curl, as wget on the ac68u and optware were not compiled with https support.

You can use ryzhov_al's guide here http://forums.smallnetbuilder.com/showthread.php?t=9449

In step 1, instead use this command: ipkg install bash libcurl sed privoxy libiconv

You might have issues on your next logons since bash will replace shell, I had to edit a file as noted in this post: http://www.smallnetbuilder.com/forums/showpost.php?p=100299&postcount=2

In step 3 download this file instead:

Code:
wget http://pastebin.com/raw.php?i=aHBYQcxs -O- | tr -d '\r' >privoxy-blocklist_0.2.sh
 
Last edited:
@ColinTaylor

What commands do you use to update the blacklist automatically? Same thing as the post-mount script? Do you run a cron job just like in the original tutorial?

Thanks!

UPDATE: I think I got it working with cron job and everything... thanks for all the help!
 
Last edited:
I use this:
Code:
# cat /jffs/configs/dnsmasq.conf.add
address=/0.0.0.0/0.0.0.0
addn-hosts=/tmp/mnt/VERBATIM/ASUS/hosts.blocked

Hey Colin!
Thanks for your contribution to this. Being a rookie but always learning, if I already have a dnsmasq.conf.add file do I just simply add the lines you specify, substituting my usb location of course?

Thanks for your help!
 
Hi all - I'm trying to get this working on my 68U, but get a "not found" error after the last quoted command below when working through Telnet. I'm a relative newbie at command-line stuff, so I'm perhaps missing something obvious - any assistance would be appreciated! Things I have checked:

- I have changed the address to point to my flash disk
- After a few attempts and failures, I created the directory and file referred to in the last line, and enabled full RW rights for anyone.
- To further troubleshoot, I pointed the last line at the JFFS partition itself, but also go the "not found" error with that attempt.

The JFFS partition was reformatted along the way also, but that's not the problem - I can browse the directories there, and the first command quoted below kicks out no errors.

No, it doesn't require entware. It think the guys posting it already had entware installed so those line were already in their scripts.

I use this:
Code:
# cat /jffs/configs/dnsmasq.conf.add
address=/0.0.0.0/0.0.0.0
addn-hosts=/tmp/mnt/VERBATIM/ASUS/hosts.blocked
 
Hi all - I'm trying to get this working on my 68U, but get a "not found" error after the last quoted command below when working through Telnet. I'm a relative newbie at command-line stuff, so I'm perhaps missing something obvious - any assistance would be appreciated! Things I have checked:

- I have changed the address to point to my flash disk
- After a few attempts and failures, I created the directory and file referred to in the last line, and enabled full RW rights for anyone.
- To further troubleshoot, I pointed the last line at the JFFS partition itself, but also go the "not found" error with that attempt.

The JFFS partition was reformatted along the way also, but that's not the problem - I can browse the directories there, and the first command quoted below kicks out no errors.
Are you using cat command to add to the file or editing the file? "cat" mean concatenate.



I've greatly modified ColinTaylor's script on page 2. My current script works with or without USB stick. Plug in USB stick for blacklist and whitelist, no USB stick mounted then just load the default list, downloading it to /tmp.

I've also added loop detection on whether internet is up, rather than rely on a timed sleep.

There is also a file detection for downloaded list. Presumably if you schedule reboot router, /tmp will be cleared for scheduled list update. This is to save bandwidth for those nice people hosting the scripts. We shouldn't download the whole list every time we plug/unplug the USB stick.

Finally, I've improved script ease of edit by putting constants for directory on the top.

I'll post it when I get home later. You call it from wan-start and/or post-mount.
 
Are you using cat command to add to the file or editing the file? "cat" mean concatenate.

Heh - I'm afraid you're inadvertently just getting me to expose further ignorance, seeing as I didn't know what I was using it for - I was just entering the commands, hoping for a good result, and dealing with the errors I understood.

I'll post it when I get home later. You call it from wan-start and/or post-mount.

Thanks, I'll wait for that before continuing my fumbling!
 
Edit the file using command like this
Code:
vi /jffs/configs/dnsmasq.conf.add
how to use vi editor: http://www.lagmonster.org/docs/vi.html

make sure these 2 lines are the only thing in that file. Hold down d key to delete other lines.
Code:
address=/0.0.0.0/0.0.0.0
addn-hosts=/tmp/adblock/hosts.blocked

That is the dnsmasq configuration part. For 2nd must have part, the scripts.

First make a file called "adblock" and add the following code into it: (sorry for lack of indentation, I find pasting into vi with indentation makes it even messier)
Code:
vi /jffs/scripts/adblock
Code:
#!/bin/sh

MNTDIR=[COLOR="Red"]/tmp/mnt/dl/adblock[/COLOR]
TMPDIR=/tmp/adblock
counti=1



#  Wait for Internet to be available.
logger -t  $(basename $0) "wyxscript: Waiting for internet to come up"
while [ $counti -lt 20 ]
do
counti='expr $counti + 1'
ping -c 1 google.com
if [[ $? == 0 ]];
then



logger -t  $(basename $0) "wyxscript: Internet is now up. "

#  Adblocker Update v

if [ ! -f $TMPDIR/hosts.downloaded ];
then

mkdir /tmp/adblock
logger -t $(basename $0) "wyxscript: Downloading new blacklists"

wget -qO- \
"http://winhelp2002.mvps.org/hosts.txt" \
"http://someonewhocares.org/hosts/zero/hosts" \
"http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext&useip=0.0.0.0" \
| grep -w ^0.0.0.0 | sed $'s/\r$//' | sort -u > $TMPDIR/hosts.downloaded

else
logger -t $(basename $0) "wyxscript: Blacklist present, no need to redownload"
fi



if [ -f $MNTDIR/hosts.whitelist ];
then
logger -t $(basename $0) "wyxscript: Whitelist found, applying it. "
grep -v -f $MNTDIR/hosts.whitelist $TMPDIR/hosts.downloaded > $TMPDIR/hosts.blocked
cat $MNTDIR/hosts.blacklist | sed $'s/\r$//' >> $TMPDIR/hosts.blocked
else
logger -t $(basename $0) "wyxscript: Whitelist not found, using default filter. "
cp $TMPDIR/hosts.downloaded $TMPDIR/hosts.blocked
fi



logger -t $(basename $0) "wyxscript: Restarting dnsmasq"
chmod -R 777 $TMPDIR/
service restart_dnsmasq

#  Adblocker Update ^

break;



else
logger -t  $(basename $0) "wyxscript: Internet is not available yet...wait 30s...."
sleep 30s
fi
done
Summary of the above script:
1. Check for internet, waits 10 minutes for internet to come up, checking every 30 seconds. Make sure you can access google.com.
2. Now that internet is connected, it checks whether the filter list has been downloaded
2a. if not, download to TMPDIR
2b. if has been downloaded, log this and proceed.
3. Check if whitelist is present
3a. If it is, apply whitelist and blacklist to the filter
3b. If not, just copies downloaded list across
4. set correct permissions otherwise there is a chance Dnsmasq complains
5. restart Dnsmasq to let it read the filter list.

make adblock script executable:
Code:
chmod 777 /jffs/scripts/adblock

Now you have a choice of ways to use this script, read this and decide when this script should run.
https://github.com/RMerl/asuswrt-merlin/wiki/User-scripts

I call the script in wan-start and post-mount. To call the script, after reading "Creating scripts" section of the above guide, put the following line underneath shebang:
Code:
sh /jffs/scripts/adblock

That's it. 1 config file, 1 script + however you decide to use it.






You should see this when the script is first run:
Code:
Sep 12 16:29:09 adblock: wyxscript: Waiting for internet to come up
Sep 12 16:29:09 adblock: wyxscript: Internet is now up. 
Sep 12 16:29:09 adblock: wyxscript: Downloading new blacklists
Sep 12 16:29:12 adblock: wyxscript: Whitelist found, applying it. 
Sep 12 16:29:12 adblock: wyxscript: Restarting dnsmasq
...
Sep 12 16:29:13 dnsmasq[4521]: read /tmp/adblock/hosts.blocked - 23817 addresses
...

Also, don't forget to do scheduled reboot. The script is designed so that only on each reboot you update your block list.
https://github.com/RMerl/asuswrt-merlin/wiki/Scheduled-Reboot



Optionally you many want whitelist and blacklist.

Edit the text in red to point it to the folder your whitelist and blacklist are situated. The lists should be called hosts.whitelist and hosts.blacklist. It doesn't matter where they are, they can be in /jffs or on a USB stick, as long as red text points to there the script will find it. You are safe to put it into /jffs as these 2 files don't change often.

Example of whitelist and blacklist are in Colin's post here:
http://forums.smallnetbuilder.com/showthread.php?t=18895&page=2

Credit goes to ColinTaylor for the original script.
 
Last edited:
Thanks for the great detail - much appreciated! Something is still not quite right, as my logs indicate that nothing is running, but I'll play with it more over the weekend - I'm no doubt getting an address wrong.

Optionally you many want whitelist and blacklist.

Edit the text in red to point it to the folder your whitelist and blacklist are situated. The lists should be called hosts.whitelist and hosts.blacklist. It doesn't matter where they are, they can be in /jffs or on a USB stick, as long as red text points to there the script will find it. You are safe to put it into /jffs as these 2 files don't change often.

Example of whitelist and blacklist are in Colin's post here:
http://forums.smallnetbuilder.com/showthread.php?t=18895&page=2

Credit goes to ColinTaylor for the original script.
 
I'm sorry, I forgot to copy+paste my dnsmasq file. Make sure "/jffs/configs/dnsmasq.conf.add" file is exactly as the the 2nd quote box above. (it's been updated)

then run
Code:
service restart_dnsmasq

If the log file says something like this, it's configured correctly.
Code:
Sep 12 16:29:13 dnsmasq[4521]: read /tmp/adblock/hosts.blocked - 23817 addresses
 
If the log file says something like this, it's configured correctly.
Code:
Sep 12 16:29:13 dnsmasq[4521]: read /tmp/adblock/hosts.blocked - 23817 addresses

Thanks so much! Got it to work - I had to also
touch /tmp/adblock/hosts.blocked

as it seemed from my log that the file wasn't found, or being created (and I also chmod'ed that to 777 to be sure), but all seems to be working fine now.
 
Be sure to do a restart to check whether your scripts still works after.

I'm not familiar with purpose of touch command, if it works for you, you might want to add this into your script, somewhere above "service restart_dnsmasq"

Also be sure to setup scheduled reboots. My script uses the principle that /tmp directory is wiped after reboot to update its block lists.
 
Need help executing

Edit the file using command like this
Code:
vi /jffs/configs/dnsmasq.conf.add
how to use vi editor: http://www.lagmonster.org/docs/vi.html

make sure these 2 lines are the only thing in that file. Hold down d key to delete other lines.
Code:
address=/0.0.0.0/0.0.0.0
addn-hosts=/tmp/adblock/hosts.blocked

That is the dnsmasq configuration part. For 2nd must have part, the scripts.

First make a file called "adblock" and add the following code into it: (sorry for lack of indentation, I find pasting into vi with indentation makes it even messier)
Code:
vi /jffs/scripts/adblock
Code:
#!/bin/sh

MNTDIR=[COLOR="Red"]/tmp/mnt/dl/adblock[/COLOR]
TMPDIR=/tmp/adblock
counti=1



#  Wait for Internet to be available.
logger -t  $(basename $0) "wyxscript: Waiting for internet to come up"
while [ $counti -lt 20 ]
do
counti='expr $counti + 1'
ping -c 1 google.com
if [[ $? == 0 ]];
then



logger -t  $(basename $0) "wyxscript: Internet is now up. "

#  Adblocker Update v

if [ ! -f $TMPDIR/hosts.downloaded ];
then

mkdir /tmp/adblock
logger -t $(basename $0) "wyxscript: Downloading new blacklists"

wget -qO- \
"http://winhelp2002.mvps.org/hosts.txt" \
"http://someonewhocares.org/hosts/zero/hosts" \
"http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext&useip=0.0.0.0" \
| grep -w ^0.0.0.0 | sed $'s/\r$//' | sort -u > $TMPDIR/hosts.downloaded

else
logger -t $(basename $0) "wyxscript: Blacklist present, no need to redownload"
fi



if [ -f $MNTDIR/hosts.whitelist ];
then
logger -t $(basename $0) "wyxscript: Whitelist found, applying it. "
grep -v -f $MNTDIR/hosts.whitelist $TMPDIR/hosts.downloaded > $TMPDIR/hosts.blocked
cat $MNTDIR/hosts.blacklist | sed $'s/\r$//' >> $TMPDIR/hosts.blocked
else
logger -t $(basename $0) "wyxscript: Whitelist not found, using default filter. "
cp $TMPDIR/hosts.downloaded $TMPDIR/hosts.blocked
fi



logger -t $(basename $0) "wyxscript: Restarting dnsmasq"
chmod -R 777 $TMPDIR/
service restart_dnsmasq

#  Adblocker Update ^

break;



else
logger -t  $(basename $0) "wyxscript: Internet is not available yet...wait 30s...."
sleep 30s
fi
done

Hi

Your script works perfectly if i run it manually from jffs/script folder with WinSCP on windows, but how do i get this to execute everyday or when i reboot my router?

Sorry i am new to this so any simple guide would be greatly appreciated.

Also just came back from DD WRT firmware, that was unstable for my daily use, but had adblock set using the dd wrt command menu. On merlin fw its more of a challenge for me.
 
Hi

Your script works perfectly if i run it manually from jffs/script folder with WinSCP on windows, but how do i get this to execute everyday or when i reboot my router?

Sorry i am new to this so any simple guide would be greatly appreciated.

Also just came back from DD WRT firmware, that was unstable for my daily use, but had adblock set using the dd wrt command menu. On merlin fw its more of a challenge for me.
The second part of the post you quoted details how to call the script from Asuswrt-Merlin standard scripts.

In steps:
1. Navigate to /jffs/scripts
2. Create a text file wan-start
3. put this into wan-start
Code:
sh /jffs/scripts/adblock
4. Create scheduled reboot by following this:
https://github.com/RMerl/asuswrt-merlin/wiki/Scheduled-Reboot

wan-start will run when the internet connection is starting up. This happens during boot.

If you are using blacklist and whitelist on a USB stick, you may also want to call the script in post-mount. Repeat step 1 to 3 with post-mount.

Hope that helps :)
 
Adblock script not working for me...

This script configuration worked for me at first, but then ceased to block any ads. I'm not sure what I did wrong. Here's some excerpts from my log:

I get this error message in my log:

Dec 31 19:00:25 dnsmasq[626]: read /etc/hosts - 5 addresses
Dec 31 19:00:25 dnsmasq[626]: failed to load names from /tmp/adblock/hosts.blocked: No such file or directory

Then I get a notice from the adblock script that the Internet is up:

Dec 31 19:00:28 adblock: wyxscript: Waiting for internet to come up
Dec 31 19:00:28 adblock: wyxscript: Internet is now up.
Dec 31 19:00:28 adblock: wyxscript: Downloading new blacklists

Then this:

Dec 31 19:00:28 dnsmasq[752]: read /etc/hosts - 5 addresses
Dec 31 19:00:28 dnsmasq[752]: failed to load names from /tmp/adblock/hosts.blocked: No such file or directory

Dec 31 19:00:33 adblock: wyxscript: Whitelist not found, using default filter.
Dec 31 19:00:33 adblock: wyxscript: Restarting dnsmasq

Then it looks like things are working:

Sep 23 18:34:03 dnsmasq[831]: read /etc/hosts - 5 addresses
Sep 23 18:34:03 dnsmasq[831]: read /tmp/adblock/hosts.blocked - 23822 addresses
Sep 23 18:34:03 dnsmasq[831]: using nameserver 192.168.1.1#53

But nothing happens. Any suggestions on what I might do?

I might have the blacklist set up incorrectly -- I created a file called hosts.blacklist and this is what's in it:

# cat /tmp/mnt/sda1/adblock/hosts.blacklist
0.0.0.0 googleadservices.com
0.0.0.0 googleads.g.doubleclick.net

Could that be causing my problem?

Thanks for your help!
 
Update

I went ahead and deleted the hosts.blacklist file, and the ad-blocking seems to sort of work again. Some ads are blocked, some aren't. Youtube ads aren't blocked. A lot of Doubleclick ads get through. Also, some HTML pages are taking forever to load on my iPad's Chrome browser.
 
As long as the very last message in time have this, your block list will be working.
dnsmasq[831]: read /tmp/adblock/hosts.blocked - nnn addresses

The script doesn't appear to be reading your blacklist and whitelist, it said:
adblock: wyxscript: Whitelist not found, using default filter.
Make sure you have edited MNTDIR at the start of the adblock script (marked in red) to where you store hosts.whitelist and hosts.blacklist.


Regarding not blocking some adverts, this will be the blacklists you download. It may not be comprehensive. Try to add them to the blacklist.

I've never had problems with pages taking too long to load on a range of my devices. The few lines of log files you posted doesn't show any possible problems.
 
I went ahead and deleted the hosts.blacklist file, and the ad-blocking seems to sort of work again. Some ads are blocked, some aren't. Youtube ads aren't blocked. A lot of Doubleclick ads get through. Also, some HTML pages are taking forever to load on my iPad's Chrome browser.

Make sure you're using the router as DNS and also try running ipconfig /flushdns from a command prompt on your PC. Reboot the iPad I guess... also empty the cache of all your browsers.

I use the first method descibed in the thread and it's working fine, almost no ads getting through and speed is very good on all my devices.
 
I followed wuyanxu's steps but some of the ads on web pages show up with an "X" instead.

This is on my TMobile router running Merlin.

Are there any other good options?

I bought an Adtrap but I'm not thrilled with it and may send it back.
 

Latest threads

Sign Up For SNBForums Daily Digest

Get an update of what's new every day delivered to your mailbox. Sign up here!
Top