pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[Makaveli]

I just set this up from AB-Solution 3.0 and everything seems to be running great. I also made sure to change the port 443 that AI cloud was using before doing the install.

http://192.168.1.3/servstats

 

[RASG]

Yes, but I think the non-secure sites are working from your message above, if no image file extension detected you get a null text response by default. Many many years ago a .gif was the default, but we found Internet Explorer, for example gave error messages because it sometimes tried to execute the binary image data as javascript. So pixelserv now tries to return a null image in the file type corrected, swf, bmp etc...

i thought that PS would always replace the empty spaces / placeholders.
what you're saying is that the image below does not represent any error.
if that is the case, i really misunderstood what PS was supposed to do.

 

[mstombs]

A broken image symbol is not what is expected, but what OS/browser is that? Majority of ads now seem to be served over https connections, which might explain that, but yes the original idea of a scale 1x1 gif was to allow the browser to collapse the white space intended fo rads. I still have problem with google ad redirects, but also tend to use uBlock in browser as well.

 

[RASG]

A broken image symbol is not what is expected, but what OS/browser is that? Majority of ads now seem to be served over https connections, which might explain that, but yes the original idea of a scale 1x1 gif was to allow the browser to collapse the white space intended fo rads. I still have problem with google ad redirects, but also tend to use uBlock in browser as well.

well, im on OSX, and both firefox and chrome are giving me the same broken image symbol.
also, i imported both the certificate and the key used by PS to my web browsers.
same result.

plus, every time i reload, lets say, snbforums.com, the counter increases for the line below

slh 58 # of accepted HTTPS requests

which means, i believe, that https requests are being correctly served by PS.

 

[Popov]

Im having a hard time to get PS to work on my ac66u, and could really use some help.
The ads are not being replaced by the 1px image served from PS.
I running out of ideas...

First thing : pixelserv is only here to serve a 1*1 pixel, you need a dns to intercept the dns request and redirect them to pixelserv.
I don't see one in your recap

Second, having a blank/missing image is normal, if you block the image itself. You need to add to your dns the address of the ads source web site. For example : doubleclick.net
Blocking the image won't prevent the javascript to be loaded. Blocking the source will do both, usually.

third : did you try what @elorimer suggested ?

 

[RASG]

First thing : pixelserv is only here to serve a 1*1 pixel, you need a dns to intercept the dns request and redirect them to pixelserv.

Sometimes it is hard for me be clear when explaning something that went wrong, since i automatically assume that everybody reading my text has the same level of knowledge about the subject.
This is wrong, and i apologize.

On item 10- pinging ad domain to see who responds of my post you can verify that i already have domain redirection set.
How do i know? Because my internal IP address is responding for doubleclick.net domain.

ramon@RT-AC66U:/tmp/home/root# ping doubleclick.net

PING doubleclick.net (192.168.25.101): 56 data bytes
64 bytes from 192.168.25.101: seq=0 ttl=64 time=0.356 ms
64 bytes from 192.168.25.101: seq=1 ttl=64 time=0.272 ms

I don't see one in your recap

That is because i only covered the pixelserv installation.
But i assure you, i already have dnsmasq up and running, blocking thousands of ads.
Now, I just want to colapse the placeholders using PS.

Second, having a blank/missing image is normal, if you block the image itself.
You need to add to your dns the address of the ads source web site. For example : doubleclick.net
Blocking the image won't prevent the javascript to be loaded. Blocking the source will do both, usually.

My hosts file loaded by dnsmasq is exactly that: hosts.
I do not have any resource path in this file, only domain hosts.
Like doubleclick.net, www.doubleclick.net, ad.doubleclick.net, and so on.

third : did you try what @elorimer suggested ?

You mean accessing http://192.168.25.101/servstats ?
Yes, i did.
On my post #386 i posted the number of accepted HTTPS requests, from the stats page.

 

[Popov]

Good
Then, as you have a redirect for doubleclick, if you open a page to http://doubleclick.net/test.gif , do you get the image ? (pixeserv adapt to the request, asking for / give a blank page, asking for any image will give the 1x1 image)
If yes, you have it working correctly, the next part is handling the https peculiar ways : open https://doubleclick.net/test.gif
if you have nothing, something is wrong on pixelserv conf. Otherwise, you'll have a message about the invalid certificate, as you said you didn't import it yet

Speaking of pixelserv, having 2 process is not an anomaly, but depends of the settings when compiled. In this situation, one is answering requests, the other is here mostly for writing certs on the disk.
Also, as it is running under the user "ramon", did you set valid access rights to the directory where it should write the certificates ?

 

[RASG]

Good
Then, as you have a redirect for doubleclick, if you open a page to http://doubleclick.net/test.gif , do you get the image ? (pixeserv adapt to the request, asking for / give a blank page, asking for any image will give the 1x1 image)
If yes, you have it working correctly

Yes. Got the image when requesting one.

the next part is handling the https peculiar ways : open https://doubleclick.net/test.gif
if you have nothing, something is wrong on pixelserv conf. Otherwise, you'll have a message about the invalid certificate, as you said you didn't import it yet

First time, got an error. Looked at PS logs and found this:

Nov 16 23:25:05 pixelserv[11411]: doubleclick.net doubleclick.net missing
Nov 16 23:25:06 pixelserv[7916]: cert doubleclick.net generated and saved

So i reloaded the page, and got a different error.
Then it hit me: chrome is using the operating system keychain (im on osx).
So i opened the keychain and forced the system to trust my certificate.
Voilà! Got the image served by PS.

Speaking of pixelserv, having 2 process is not an anomaly, but depends of the settings when compiled. In this situation, one is answering requests, the other is here mostly for writing certs on the disk.
Also, as it is running under the user "ramon", did you set valid access rights to the directory where it should write the certificates ?

Ah, yes. But thanks for asking.

-rw-rw-rw-    1 ramon    root        1.9K Nov 15 19:34 _.g.doubleclick.net
-rw-rw-rw-    1 ramon    root        1.9K Nov 15 18:58 _.google-analytics.com
-rw-rw-rw-    1 ramon    root        1.9K Nov 15 19:36 _.googleadservices.com
-rw-rw-rw-    1 ramon    root        1.9K Nov 15 19:00 _.googlesyndication.com
-rw-rw-rw-    1 ramon    root        1.9K Nov 15 19:04 _.googletagmanager.com

After checking that PS is serving the image both on http and https when requested, i have to inform that, sadly, its still not removing the placeholders...

But i really, really wanna thank you for your time [wasted] trying to help me.

Will try again on the next update.

 

[Popov]

After checking that PS is serving the image both on http and https when requested, i have to inform that, sadly, its still not removing the placeholders...

pixelserv can't remove them : it's part of the page. Only a plugin in the browser is able to do so, like adblock, and might not always be successful.
Using pixelserv prevent sites to load additionnal ressources, but if the page itself has a placeholder designed, it'll stay. The main point is having a global solution for all your devices at home, without the need to install a plugin into each browser.

This said, for your current issue, I usualy use the network tab in the dev tools panel to track external sites addresses.
Most of the time, when you have different sites used for the same ads, it's a chain with one external domain initializing the area and loading the content from another domain. Blocking the first in the chain is enough, and you end with the whole block removed, or in the worst case an area staying blank (but whitout the shape like you have)
The case you have can still happen, but still very rare (mostly a site serving ads from one of his subdomains)

 

[jrmwvu04]

Now, I just want to colapse the placeholders using PS.

This will not be possible with only pixelserv, as it doesn't modify the HTML beyond a pixel replacing some other element. That is, it will not format any whitespace. The easiest way to accomplish that is to use a browser extension.

Edit: well, seems Popov had that covered.

 

[mstombs]

I use dnsmasq's domain blocking feature to block all subdomains. PGL created a simple download in correct format many years ago, and I still use it! https://pgl.yoyo.org/adservers/news.php#dnsmasq-linksys-tomato-local-blocking-and-you

 

[RASG]

I use dnsmasq's domain blocking feature to block all subdomains. PGL created a simple download in correct format many years ago, and I still use it! https://pgl.yoyo.org/adservers/news.php#dnsmasq-linksys-tomato-local-blocking-and-you

yes. they also accept any ip address as parameter.
in my script i have, for example

BLACKHOLE="192.168.25.101"

WGET="wget -b -N -nv --show-progress --no-hsts --output-file=/tmp/wget.log"

$WGET "http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext&useip=$BLACKHOLE"

a very nice feature

 

[mstombs]

You mean I don't need the "sed"? I use the dnsmasq domain block format

#!/bin/sh
CNF=/mnt/usb4gb/adblock.conf
WHL=/mnt/usb4gb/whitelist
ADS="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext"
AIP="192.168.66.254" # changes 127.0.0.1 to this globally
wget -O - "$ADS"|sed "s/127.0.0.1/$AIP/" | grep -Fvf $WHL >$CNF

yes it works

ADS="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=dnsmasq&showintro=0&mimetype=plaintext&useip=$AIP"
wget -O - "$ADS"|grep -Fvf $WHL >$CNF

 

[MrT69]

Hi folks. Is someone here who is able to help to compile this code for FreeBSD and LINUX?
I have had no probs to do it with the original code from hunterZ but it seems this code is some kind of tricky.

Can someone help?

Million thanks in advance!

 

[kvic]

The additional difference is the linking to openssl and pthread libraries. Once you have that setup in your build environment, Makefile shall just work in Linux (and Freebsd too I believe). What errors were you running into...?

 

[kvic]

Hi folks. Is someone here who is able to help to compile this code for FreeBSD and LINUX?
I have had no probs to do it with the original code from hunterZ but it seems this code is some kind of tricky.

Can someone help?

Million thanks in advance!

I uploaded the binary for 64-bit x86 Linux. Check out this page.

My environment for 32-bit x86 Linux is broken from the beginning. Nor I have Freebsd environment. Sorry..

 

[mstombs]

I remember a problem in the past with bsd and Linux socket options, don't think we bother with tweaking linger2 any more

https://forums.freebsd.org/threads/26786/

 

[thelonelycoder]

Hey @kvic, I noticed because a user provoked me to test my switches option in AB-Solution, PS version ki:
-k https_port (443 if omitted)
-p http_port (80 if omitted)
If only one port is set, say -p 8080, only this port is started, https will not, although is should according to the description.
You need to specify both ports if one is set.

Is this intended and correct behavior for ki?

Edit, the output added:

tlc@RT-AC87U:/tmp/home/root# pixelserv-tls --help
Usage:pixelserv-tls
    ip_addr/hostname (all if omitted)
    -2 (disable HTTP 204 reply to generate_204 URLs)
    -f (stay in foreground - don't daemonize)
    -k https_port (443 if omitted)
    -l (log access to syslog)
    -n i/f (all interfaces if omitted)
    -o select_timeout (10 seconds)
    -p http_port (80 if omitted)
    -r (deprecated - ignored)
    -R (disable redirect to encoded path in tracker links)
    -s /relative_stats_html_URL (/servstats if omitted)
    -t /relative_stats_txt_URL (/servstats.txt if omitted)
    -u user ("nobody" if omitted)
    -z path_to_https_certs (/opt/var/cache/pixelserv if omitted)

 

[mstombs]

While "-k" only exists in this fork the original behaviour has been continued - default ports for both http and https are only applied if no ports defined on the command line, agree description a bit misleading.


https://github.com/kvic-z/pixelserv...b67c161f96da6ad6e3c042b8cae2/pixelserv.c#L335

I currently use

pixelserv-tls version: v35.HZ12.Ki compiled: Oct 30 2016 09:36:51 options: 192.168.66.254 -p 80 -p 81 -p 8080 -p 8081 -k 443 -o 2

 

[thelonelycoder]

While "-k" only exists in this fork the original behaviour has been continued - default ports for both http and https are only applied if no ports defined on the command line, agree description a bit misleading.

https://github.com/kvic-z/pixelserv-tls/blob/master/pixelserv.c#L335

I currently use

pixelserv-tls version: v35.HZ12.Ki compiled: Oct 30 2016 09:36:51 options: 192.168.66.254 -p 80 -p 81 -p 8080 -p 8081 -k 443 -o 2

Thanks for the clarification.
I've added this info to the PS switches description in the upcoming AB3.1.