pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[kvic]

I configured a custom adblock with dnsmasq but I do not know how to remove this warning here in the Forum. Any tips?

I like to take a poll.

Given a domain e.g. snbforum.com, all ads/trackers on that domain will allow to pass through i.e. not blocked.

How popular/useful will this feature be in your daily use (other than snbforum)?

Feel free to let me know your opinion though not that such as a feature is coming nor promised.

 

[DonnyJohnny]

I like to take a poll.

Given a domain e.g. snbforum.com, all ads/trackers on that domain will allow to pass through i.e. not blocked.

How popular/useful will this feature be in your daily use (other than snbforum)?

Feel free to let me know your opinion though not that such as a feature is coming nor promised.

I understand the purpose of your proposal. It would be useful for site using anti Adblock script. But....

Ask question.
I whitelist snbforum.com, whatever with snbforum.com will whitelist. But I visit other domain and I want those trackers/ad that snbforum used to be blocked as usual. Is that how u want it to work?

 

[XIII]

How popular/useful will this feature be in your daily use (other than snbforum)?

I don't use whitelists in ad blockers, so for me it would not be useful.

(I'd rather pay a content provider a reasonable fee, than an advertiser)

 

[kvic]

I whitelist snbforum.com, whatever with snbforum.com will whitelist. But I visit other domain and I want those trackers/ad that snbforum used to be blocked as usual. Is that how u want it to work?

Exactly.

 

[Asad Ali]

Exactly.

+1 from my side if you ever execute this idea.

 

[Martineau]

@kvic, just a quick question regarding the pixelserv-tls initialisation.

I recorded the HTOP screen for the first 15 mins of the boot process, and as expected it takes roughly 10-12 mins for the router to finally 'settle' with regards to CPU usage.

Strangely the video shows pixelserve-tls ranked 1st in almost every HTOP reporting interval - consuming anywhere between 5%-30% CPU and narrowly tops the table of tasks consuming the most CPU

Here is a summary snapshot taken approx. 16 mins after the physical reboot:

40.64 CPU seconds pixelserv-tls
40.37 CPU seconds dnsmasq
38.90 CPU seconds nt_center
17.44 CPU seconds htop

I'm not suggesting that this causes any detrimental effect, but I was just curious as to what pixelserv-tls is actually doing during the boot process?

 

[kvic]

I'm not suggesting that this causes any detrimental effect, but I was just curious as to what pixelserv-tls is actually doing during the boot process?

Nothing particular with respect to boot process. Generally on the process startup, pixelserv-tls initializes itself very quickly and then sleeps and waits for incoming requests.

I would think something wrong in your setup. Enable '-l 5' and check the log which may shed some light on the problem.

Below is my accumulated cpu time..about 36s after being up for 7 days and having processed 25K requests. So on average it's about 14ms per request. The numbers demonstrate pixelserv-tls is a superb efficient daemon.

It runs on my SBC..equivalent to 86U CPU wise, slower clock but two more cores.

 

[rgnldo]

I like to take a poll.

Given a domain e.g. snbforum.com, all ads/trackers on that domain will allow to pass through i.e. not blocked.

How popular/useful will this feature be in your daily use (other than snbforum)?

Feel free to let me know your opinion though not that such as a feature is coming nor promised.

Ok! I had solved this detail. Releasing the whole site is not legal. Release some snbforums.com advertisers

 

[slobodan]

Ok! I had solved this detail. Releasing the whole site is not legal. Release some snbforums.com advertisers

Why is it not legal? According to the laws of which country?

kazoo.ga seems down.

 

[kvic]

kazoo.ga seems down.

Fixed. Up running..

 

[Martineau]

I would think something wrong in your setup. Enable '-l 5' and check the log which may shed some light on the problem.

It would appear that during the boot period, pixelserv-tls successfully executes 'sslctx_tbl_load:' for 205 items ('_.amazon.com' to 'zdbb.net') then apparently repeats the same cycle a couple of times again?

NOTE: Three devices (all Amazon Echo) finally results in uce=928 in the 15 min boot window.

handshake failed: unknown cert. xxx.xxxx.xxx.xxx:ppppp server device-metrics-us-2.amazon.com
handshake failed: unknown cert. xxx.xxxx.xxx.xxx:ppppp server device-metrics-us.amazon.com

Spoiler: pixelserv-tls log

Dec 13 18:07:29 RT-AC68U pixelserv-tls[4699]: pixelserv-tls 2.2.1-rc.4 (compiled: Dec  9 2018 19:49:24 flags: no_tls1_3) options: 10.88.8.3 -l 5
Dec 13 18:07:29 RT-AC68U pixelserv-tls[4699]: sslctx_tbl_load: _.amazon.com
<snip>
Dec 13 18:07:31 RT-AC68U pixelserv-tls[4699]: sslctx_tbl_load: zdbb.net
Dec 13 18:07:31 RT-AC68U pixelserv-tls[4699]: Listening on :10.88.8.3:443
Dec 13 18:07:31 RT-AC68U pixelserv-tls[4699]: Listening on :10.88.8.3:80
Dec 13 18:08:39 RT-AC68U pixelserv-tls[4699]: handshake failed: unknown cert. client 10.88.8.18:52022 server device-metrics-us-2.amazon.com
<snip>
Dec 13 18:08:48 RT-AC68U pixelserv-tls[4699]: handshake failed: unknown cert. client 10.88.8.18:33340 server device-metrics-us-2.amazon.com
Dec 13 18:09:07 RT-AC68U pixelserv-tls[4699]: no file extension /js from path /gtag/js
Dec 13 18:09:07 RT-AC68U pixelserv-tls[4699]: 10.88.8.114 www.googletagmanager.com GET /gtag/js?id=UA-58643-34 HTTP/1.1 tls_1.2
Dec 13 18:09:11 RT-AC68U pixelserv-tls[4699]: no file extension /js from path /gtag/js
Dec 13 18:09:11 RT-AC68U pixelserv-tls[4699]: 10.88.8.114 www.googletagmanager.com GET /gtag/js?id=UA-58643-34 HTTP/1.1 tls_1.2
Dec 13 18:10:16 RT-AC68U pixelserv-tls[4699]: 167 uts, 5 log, 1 kcc, 1 kmx, 0.00 kvg, 0 krq, 234 req, 370 avg, 370 rmx, 96 tav, 193 tmx, 2 slh, 0 slm, 0 sle, 0 slc, 232 slu, 0 v13, 2 v12, 0 v10, 0 uca, 0 ucb, 232 uce, 0 ush, 206 sct, 233 sch, 0 scm, 0 scp, 0 sst, 0 ssh, 0 ssm, 0 ssp, 2 nfe, 0 gif, 0 ico, 0 txt, 0 jpg, 0 png, 0 swf, 0 sta, 0 stt, 0 ufe, 0 opt, 0 pst, 0 hed, 0 rdr, 0 nou, 0 pth, 0 204, 0 bad, 0 tmo, 0 cls, 0 cly, 0 clt, 0 err
Dec 13 18:10:16 RT-AC68U pixelserv-tls[4699]: exit on SIGTERM
Dec 13 18:14:01 RT-AC68U pixelserv-tls[2454]: pixelserv-tls 2.2.1-rc.4 (compiled: Dec  9 2018 19:49:24 flags: no_tls1_3) options: 10.88.8.3 -l 5
Dec 13 18:07:29 RT-AC68U pixelserv-tls[4699]: sslctx_tbl_load: _.amazon.com
<snip>
Dec 13 18:14:06 RT-AC68U pixelserv-tls[2454]: sslctx_tbl_load: zdbb.net
Dec 13 18:14:06 RT-AC68U pixelserv-tls[2454]: Listening on :10.88.8.3:443
Dec 13 18:14:06 RT-AC68U pixelserv-tls[2454]: Listening on :10.88.8.3:80
Dec 13 18:15:39 RT-AC68U pixelserv-tls[2454]: handshake failed: unknown cert. client 10.88.8.17:55916 server device-metrics-us.amazon.com
Dec 13 18:15:41 RT-AC68U pixelserv-tls[2454]: 10.88.8.114 wpad.martineau.lan GET /wpad.dat HTTP/1.1 tls_none
Dec 13 18:15:41 RT-AC68U pixelserv-tls[2454]: Exit recv loop socket:12 rv:269 errno:0 num_req:1
Dec 13 18:15:41 RT-AC68U pixelserv-tls[2454]: handshake failed: unknown cert. client 10.88.8.17:39258 server device-metrics-us.amazon.com
Dec 13 18:15:41 RT-AC68U pixelserv-tls[2454]: handshake failed: unknown cert. client 10.88.8.17:55323 server device-metrics-us.amazon.com
Dec 13 18:15:41 RT-AC68U pixelserv-tls[2454]: handshake failed: unknown cert. client 10.88.8.17:49124 server device-metrics-us.amazon.com
Dec 13 18:15:41 RT-AC68U pixelserv-tls[2454]: handshake failed: unknown cert. client 10.88.8.17:54241 server device-metrics-us.amazon.com
Dec 13 18:15:41 RT-AC68U pixelserv-tls[2454]: recv() ECONNRESET: Success
Dec 13 18:15:41 RT-AC68U pixelserv-tls[2454]: Exit recv loop socket:12 rv:0 errno:0 num_req:1
Dec 13 18:15:45 RT-AC68U pixelserv-tls[2454]: 10.88.8.114 wpad GET /wpad.dat HTTP/1.1 tls_none
Dec 13 18:15:46 RT-AC68U pixelserv-tls[2454]: handshake failed: unknown cert. client 10.88.8.17:46664 server device-metrics-us.amazon.com
Dec 13 18:15:46 RT-AC68U pixelserv-tls[2454]: 10.88.8.114 wpad GET /wpad.dat HTTP/1.1 tls_none
Dec 13 18:15:46 RT-AC68U pixelserv-tls[2454]: 10.88.8.114 wpad GET /wpad.dat HTTP/1.1 tls_none
<snip>
Dec 13 18:16:48 RT-AC68U pixelserv-tls[2454]: handshake failed: unknown cert. client 10.88.8.16:44582 server device-metrics-us.amazon.com
Dec 13 18:16:48 RT-AC68U pixelserv-tls[2454]: 167 uts, 5 log, 3 kcc, 3 kmx, 1.00 kvg, 1 krq, 936 req, 232 avg, 268 rmx, 5 tav, 22 tmx, 0 slh, 0 slm, 0 sle, 1 slc, 928 slu, 0 v13, 0 v12, 0 v10, 0 uca, 0 ucb, 928 uce, 0 ush, 206 sct, 930 sch, 0 scm, 0 scp, 0 sst, 0 ssh, 0 ssm, 0 ssp, 0 nfe, 0 gif, 0 ico, 0 txt, 0 jpg, 0 png, 0 swf, 0 sta, 0 stt, 7 ufe, 0 opt, 0 pst, 0 hed, 0 rdr, 0 nou, 0 pth, 0 204, 0 bad, 0 tmo, 1 cls, 0 cly, 0 clt, 0 err
Dec 13 18:16:48 RT-AC68U pixelserv-tls[2454]: exit on SIGTERM
Dec 13 18:16:49 RT-AC68U pixelserv-tls[22188]: pixelserv-tls 2.2.1-rc.4 (compiled: Dec  9 2018 19:49:24 flags: no_tls1_3) options: 10.88.8.3 -l 5
Dec 13 18:16:49 RT-AC68U pixelserv-tls[22188]: sslctx_tbl_load: _.amazon.com
<snip>
Dec 13 18:16:52 RT-AC68U pixelserv-tls[22188]: sslctx_tbl_load: zdbb.net
Dec 13 18:16:52 RT-AC68U pixelserv-tls[22188]: Listening on :10.88.8.3:443
Dec 13 18:16:52 RT-AC68U pixelserv-tls[22188]: Listening on :10.88.8.3:80

P.S. Previously you have stated that this is acceptable/normal? - although after only 15 hours I assume my count is a tad high!

uce   225273  slu break-down: # of unknown cert reported by clients

EDIT (After moderator approval): Both domains now whitelisted uce=0 during boot window, and pixelserv-tls consumed 1.87 CPU seconds!

Happy days!

 

[kvic]

It would appear that during the boot period, pixelserv-tls successfully executes 'sslctx_tbl_load:' for 205 items ('_.amazon.com' to 'zdbb.net') then apparently repeats the same cycle a couple of times again?

Your init script or someone else restarts pixelserv-tls multiple times on router startup..

P.S. Previously you have stated that this is acceptable/normal? - although after only 15 hours I assume my count is a tad high!

uce   225273  slu break-down: # of unknown cert reported by clients

In my daily usages, slu is about 10-15% of req, the total # of requests.

The actual percentage will vary with environment, usages and blacklists. But you may take mine as a yardstick..

Regardless the %, nothing people could do about it as explained before. It doesn't hurt. Some people chose to send such domains to a second instance of pixelserv-tls.

My latest is to use "TLS report" to capture new breaks that might worth a closer look. Some people in this thread have been running it for a long while. Ping me if you're interested..

EDIT:

EDIT (After moderator approval): Both domains now whitelisted uce=0 during boot window, and pixelserv-tls consumed 1.87 CPU seconds!

I have both domains blocked. Looks like some devices on your LAN are mad about it. I won't whitelist them. You may consider sending both to 0.0.0.0 for example.

 

[rgnldo]

Why is it not legal? According to the laws of which country?

Script Diversion makes partial blocking of snbforums.com. I just copied their white list.

 

[pattiri]

When I saw this in a web site; a smile appears in my face which means pixelserv-tls is working for me

 

[Cefeon]

Does anyone know the best way to install 2.2.0? The current entware package still points to 2.1.2.

I'm hoping to run the latest non-beta version. Thanks!

 

[slobodan]

Script Diversion makes partial blocking of snbforums.com. I just copied their white list.

That's not an answer to my questions. Where would that be illegal and why? I live in the Netherlands, I don't care about German laws. Or about Chinese laws. None of my business. I don't go to Saudi Arabia because I would get the death penalty for witchcraft.

 

[cmkelley]

That's not an answer to my questions. Where would that be illegal and why? I live in the Netherlands, I don't care about German laws. Or about Chinese laws. None of my business. I don't go to Saudi Arabia because I would get the death penalty for witchcraft.

I'm gonna go with a translation problem here. I suspect "not legal" is not what he meant, more like "not possible". It makes more sense that way - he was saying he couldn't simply whitelist all advertisers called by a given website, he had to whitelist them one at a time.

They tell me English is a very hard language to learn as a second language, and any language is tough if you're not able to use it every day.

 

[kvic]

Does anyone know the best way to install 2.2.0? The current entware package still points to 2.1.2.

I'm hoping to run the latest non-beta version. Thanks!

Yes, you can.

Try the following one liner in your SSH shell:

_prod=1 sh -c "$(wget -qO - https://kazoo.ga/pixelserv-tls/install-beta.sh)"

 

[elorimer]

'm gonna go with a translation problem here. I suspect "not legal" is not what he meant, more like "not possible". It makes more sense that way - he was saying he couldn't simply whitelist all advertisers called by a given website, he had to whitelist them one at a time.

I agree with you, although I think he was saying that just whitelisting snbforums doesn't work to allow add through on the site, you have to whitelist them individually, and later he said he copied diversion's white list to do that.

I think @kvic's idea of selective whitelisting is very interesting. Allowing ads on this site without allowing them on every site.

Also, my slu is about 95% of my req. Devices banging away to phone home. This isn't a problem; I think it shows how effective a solution Diversion is in sending these things on to pixelserv, and pixelserv efficiently dropping them.

 

[Cefeon]

Yes, you can.

Try the following one liner in your SSH shell:

_prod=1 sh -c "$(wget -qO - https://kazoo.ga/pixelserv-tls/install-beta.sh)"

Thanks @kvic much appreciated!