pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[kvic]

Hi kvic, is there a list of steps I can follow for installing pixelsrv-tls (and the related hosts I would lie to block) in DD-WRT? I haven't found a guide/tutorial for this platform and I'd really appreciate any contribution/help

Start with installing Entware-ng and then 'opkg install pixelserv-tls'. You shall be half way done.

Personally I haven't run DD-WRT. The guide suggested by mstombs seems very detailed to start with. I think you can substitue the generation of host files by utilising swetoast's ublockr scripts (maybe with slight modification).

You'll be the first one able to prepare a tutorial for DD-WRT users

 

[thelonelycoder]

Hey @kvic
I have released a beta version of AB-Solution with pixelserv-tls auto-install. It all works well, thank you very much.

On the cert front I am at a loss though what to do to make the https work right.
I went off of the Generating OpenVPN keys using Easy RSA tutorial.
Then generated the pixelserv.pem:
cat server.crt server.key > /jffs/etc/pixelserv.pem
And added the ca.crt and ca.key to /opt/var/cache/pixelserv

pixelserve works as advertised now:

Aug 10 16:31:27 pixelserv[1052]: www.google-analytics.com _.google-analytics.com missing
Aug 10 16:31:28 pixelserv[1053]: cert _.google-analytics.com generated and saved

What I am unsure of: From your perspective is this the correct way to get the https part up and running by having these 3 certificates (pixelserv.pem, ca.crt and ca.key) in the respective locations?
TIA

 

[kvic]

Hey @kvic
I have released a beta version of AB-Solution with pixelserv-tls auto-install. It all works well, thank you very much.

On the cert front I am at a loss though what to do to make the https work right.
I went off of the Generating OpenVPN keys using Easy RSA tutorial.
Then generated the pixelserv.pem:
cat server.crt server.key > /jffs/etc/pixelserv.pem
And added the ca.crt and ca.key to /opt/var/cache/pixelserv

pixelserve works as advertised now:

Aug 10 16:31:27 pixelserv[1052]: www.google-analytics.com _.google-analytics.com missing
Aug 10 16:31:28 pixelserv[1053]: cert _.google-analytics.com generated and saved

What I am unsure of: From your perspective is this the correct way to get the https part up and running by having these 3 certificates (pixelserv.pem, ca.crt and ca.key) in the respective locations?
TIA

Great to see pixelserv integrated in your adblock solution! That's about right on the router side. Just note a few things:

1. ca.crt and ca.key preferably generated with a lifetime of 10 yrs. Check if that's the default in the OpenVPN tutorial you quoted. Regardless, certificates generated automatically by pixelserv-tls have a lifetime of 10yrs. So with ca.crt 10 yrs the whole setup will last as long as that without human intervention.

2. server.crt, server.key and /jffs/etc/pixelserv.pem are not required for pixelserv-tls to operate. If you have users switching WebUI to HTTPS, then the server.crt/server.key might be re-used for the WebUI httpd. Check here for details: http://www.snbforums.com/threads/pi...ebserver-for-adblock.26114/page-9#post-252339

3. Personally I don't run WebUI over HTTPS. Responses are noticeably slower than plain HTTP. Beginning with recent stock/merlin firmwares (I forgot the exact version), WebUI httpd is no longer bound to all interfaces. Hence, you can create a virtual interface for pixelserv-tls e.g. I've been using these two lines in wan-start:

ifconfig br0:pixelserv 192.168.1.3 up
logger -t $tag "br0:pixelserv 192.168.1.3 created."

Then start pixelserv on 192.168.1.3 (that you will change in Entware's init script for pixelserv-tls), and have 192.168.1.3 in your host files.

Both #2 and #3 are fine. So pick either one where you see fit.

On the client side (MacOS/iOS/android/Windows), users will need to import ca.crt by following steps here: https://github.com/kvic-z/pixelserv-tls#import-cacrt-into-clients

 

[thelonelycoder]

Code:
ifconfig br0ixelserv 192.168.1.3 up
logger -t $tag "br0ixelserv 192.168.1.3 created."

That's the code I was looking for! I never liked the firewall rule route, so that will be the way to go for AB-Solution.
Glad you provided an elegant way to switch back to http Web authentication. It causes more headaches than it solves things. Firefox 48, for example, does not like the cert and an exception cannot be set anymore.

1. ca.crt and ca.key preferably generated with a lifetime of 10 yrs. Check if that's the default in the OpenVPN tutorial you quoted. Regardless, certificates generated automatically by pixelserv-tls have a lifetime of 10yrs. So with ca.crt 10 yrs the whole setup will last as long as that without human intervention.

Yes, CA_EXPIRE=3650 is the default setting for easyrsa.

Again, thanks a lot, this very good news for AB-Solution.
Now, if I could only fully automate the certificate creation.
I guess users will have to first change the variables in the easy-rsa vars file before I can semi-automate the creation.

Edit: Quotes apparently transform to !

 

[mstombs]

You can still use Firefox 48 with Asuswrt-Merlin (.59 on N66) web gui over https, but you now have to confirm an exception (my certs generated 4 months ago, now have >130 ad sites certs generated). Firefox says the certificate is invalid because "This web site does not supply ownership information." Same problem with https pixels i.e. https://doubleclick.net/ . Google Chrome still OK but is time running out?

I didn't know asuswrt-merlin now only listens on the specific lan ip address - seems to be a change in httpd source code to attach to a specific interface in

Merge with Asus GPL 380_2697
RMerl committed on 4 Apr 2016

https://github.com/RMerl/asuswrt-me...fc87930/release/src/router/httpd/httpd.c#L316

 

[thelonelycoder]

@mstombs Confirming alone does not save the exeption in my case.

 

[kvic]

Firefox says the certificate is invalid because "This web site does not supply ownership information." Same problem with https pixels i.e. https://doubleclick.net/ . Google Chrome still OK but is time running out?

How to reproduce this warning? I can't with FF 48 + Win 10 Pro. Looks fine for me:

The case of WebUI HTTPS might be a different issue.

I didn't know asuswrt-merlin now only listens on the specific lan ip address - seems to be a change in httpd source code to attach to a specific interface in

Merge with Asus GPL 380_2697
RMerl committed on 4 Apr 2016

https://github.com/RMerl/asuswrt-me...fc87930/release/src/router/httpd/httpd.c#L316

There was a guy who discovered a "security issue" on WebUI being accessible from WAN when firewall was turned off. Some members of this community regarded him as a pundit. Anyway his effort led to user benefits unrelated to the original security issue. lol

 

[kvic]

Now, if I could only fully automate the certificate creation.
I guess users will have to first change the variables in the easy-rsa vars file before I can semi-automate the creation.

Some aspects of EasyRSA can be controlled through environment variables. Configurable from a text file or from command line right before execute EasyRSA command. E.g

export EASYRSA_KEY_SIZE=1024
./build-ca

This shall generate a CA cert with key length 1024 bit. Note that minimal key length is good for pixelserv. Save computation on both clients and the router. Look at vars.example in EasyRSA distro for all available variables.

Alternatively you get rid of EasyRSA dependence of altogether. Use openssl command directly. Since we're only generating one CA cert, shall be quick to figure out the exact arguments.

I could have added the option to generate a CA cert in pixelserv-tls. To avoid any suspicion on security compromise, I opted not to do that. lol

 

[thelonelycoder]

I could have added the option to generate a CA cert in pixelserv-tls. To avoid any suspicion on security compromise, I opted not to do that. lol

It's never too late mate.
This could be handy to be an option in ARGS=""
I would really like that.

 

[mstombs]

You are correct different issue between router gui and pixelserv. It seems my router has forgotten to use the pixelserv certificate, and you can either grant an exception on each use, or store the certificate in Firefox advanced setup. The doubleclick one is different however, no option to ignore security issue or save/import cert that I can find....

 

[kvic]

It's never too late mate.
This could be handy to be an option in ARGS=""
I would really like that.

Okay, I'll consider that

For the time being, you can use the following openssl commands to directly generate a pair of ca.key/ca.crt in your scripts:

cd /opt/var/cache/pixelserv
openssl genrsa -out ca.key 1024
openssl req -key ca.key -new -x509 -days 3650 -sha256 -extensions v3_ca -out ca.crt -subj "/CN=Pixelserv CA"

This will generate a 1024-bit CA cert with a lifetime of 10 yrs.

 

[kvic]

You are correct different issue between router gui and pixelserv. It seems my router has forgotten to use the pixelserv certificate, and you can either grant an exception on each use, or store the certificate in Firefox advanced setup. The doubleclick one is different however, no option to ignore security issue or save/import cert that I can find....

Should not require to import/grant exception to individual certs. Only the CA cert need import once. FF has its own repository of CA certs. Hence, you need to do it once specifically for FF. Otherwise, import into OS once is sufficient. https://github.com/kvic-z/pixelserv-tls#windows

Good news is that beginning with FF 49, an experimental feature is added to automatically import from OS repository once it's added there. No longer require separate manual import for FF. Seems this feature shall be in FF on day one when they decided to have separate CA cert repository.

 

[mstombs]

I may have lost settings in Firefox due to a reinstall? I have successfully re-imported the Pixelserv CA cert for use in Chrome and Firefox on 2 different Win7 laptops, but I am unable to use the same cert for the https web gui and get green padlock in browser. Using router self generated I cert can import the cert and get green padlock when accessing using the lan IP address - the cert is generated with that IP encoded it seems. So I am now unsure if I ever got the https web gui working with pixelserv CA? You cannot copy the script on http://www.snbforums.com/threads/pi...ebserver-for-adblock.26114/page-9#post-252339 in one go because of the "$" signs - maybe I did it wrong! I definitely remember noticing the https web gui works much smoother when you get the security approved with a green padlock!

 

[kvic]

@mstombs Don't get lost in illusions. I verified that when I posted the method. So your memory was correct. Something could have changed in FF or/and httpd. I stopped using FF on daily basis long time ago. Based on feedback in this thread FF seems to be the frontrunner on enforcing new security measures... You'll be miles better off switch back to plain HTTP WebUI. Lots more responsive.

 

[shooter40sw]

Hi guys, what lines do I have to change to get this working with ab solutions?, I just installed pixelserve, and made @kvic
ifconfig br0ixelserv 192.168.1.3 up
logger -t $tag "br0ixelserv 192.168.1.3 created."

The last line stalls, so I just went with the first line, and serves the pixel, I know I need to replace some IP address but not very good at code so I dont know the ones to change to the new 192.168.1.3, thanks

 

[thelonelycoder]

The last line stalls, so I just went with the first line, and serves the pixel, I know I need to replace some IP address but not very good at code so I dont know the ones to change to the new 192.168.1.3, thanks

All IPv4 addresses in /adblocking/hosts-adblock and blacklist.txt.
BTW I'm working on ab-solution beta3 that will take advantage of this.

 

[shooter40sw]

All IPv4 addresses in /adblocking/hosts-adblock and blacklist.txt.
BTW I'm working on ab-solution beta3 that will take advantage of this.

I Know you are working on it , just wanted to take a test drive before, I plan de to factory reset the whole router and do everything from scratch as soon as you release, but I wanted to know in what line of code do I replace and place the new IP when I run the script.
Thanks!

 

[shooter40sw]

And another thing, Im point it to another port the 8080, and my GUI is sitting on 80, but pixelserv is working cause I can point directly to the IP and port 8080 and there is nothing there, also the stats page works, but I changed manually a blacklisted page to the router IP and I the error page that it could not be reached, what Im I missing? thanks
And if I run a nslookup on pricegrabber.com, that the one I blacklisted with the new IP, and the router points to the IP that I placed, it just does not go to the 8080 port

 

[mstombs]

By default the C pixelserv listens on port 80 and port 443 for https, because the dns diverts the IP address, and the web pages use the scripted or default IPs. I came across something using port 81 and 8080 so I get my pixelserv to listen on these as well with custom options:-

pixelserv-tls version: V35.HZ12.Kh compiled: May 11 2016 15:13:54 options: 192.168.66.254 -p 80 -p 81 -p 8080 -p 8081 -k 443 -o 2
4995871 uts, 5682573 req, 1284 avg, 34331 rmx, 109 tav, 4159 tmx, 0 err, 8562 tmo, 5625639 cls, 0 nou, 0 pth, 12704 nfe, 2263 ufe, 492 gif, 0 bad, 18294 txt, 60 jpg, 76 png, 18 swf, 18 ico, 5579788 slh, 84 slm, 0 sle, 2557 slu, 11 sta, 3 stt, 0 204, 14114 rdr, 277 pst, 0 hed, 0 log

 

[shooter40sw]

By default the C pixelserv listens on port 80 and port 443 for https, because the dns diverts the IP address, and the web pages use the scripted or default IPs. I came across something using port 81 and 8080 so I get my pixelserv to listen on these as well with custom options:-

pixelserv-tls version: V35.HZ12.Kh compiled: May 11 2016 15:13:54 options: 192.168.66.254 -p 80 -p 81 -p 8080 -p 8081 -k 443 -o 2
4995871 uts, 5682573 req, 1284 avg, 34331 rmx, 109 tav, 4159 tmx, 0 err, 8562 tmo, 5625639 cls, 0 nou, 0 pth, 12704 nfe, 2263 ufe, 492 gif, 0 bad, 18294 txt, 60 jpg, 76 png, 18 swf, 18 ico, 5579788 slh, 84 slm, 0 sle, 2557 slu, 11 sta, 3 stt, 0 204, 14114 rdr, 277 pst, 0 hed, 0 log

Thanks, I got something working with the port 80, without changing the GUI, so now it pixelserve listens on the 80 and 443 but on another IP - 192.168.1.3, This with kvics suggestion to lonely. but I dont know what lines on the ab solutions I need to change so that the final host file is 192.168.1.3 whatever.com and so on, Im looking on the lonelys code but Im , cant figure out yet what to change