pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[kvic]

I keep opening phone apps now to see what else I can squash, thanks to your guidance. Then I will do the same with the other two mobile devices.

So far we assume exceptions like this are minimal. Hopefully the list in hosts.add (or other means) won't grow to a point not manageable.

Let's see how it goes with more users trying "privacy breach inspection" In a future version of pixelserv-tls, perhaps we can build in better suppression control if necessary.

Thanks for being among the forerunners on trying new stuff.

 

[penguin22]

I think you've brought up a good point on the side of supporting vendors doing fingerprint check in order to protect users from their systems being compromised with 'dubious' or even legitimate CA certs.

I went through both macOS and Windows many times in order to have a user CA cert installed. The security check that has to go through is stringent to a point that people may not want to repeat it casually. But there is no way to avoid a legitimate CA erroneously (or purposely) issue a wrong server certificate.

But I have the suspicion (that can't get rid of) that big (and small) guys are doing so for multiple reasons, the primary is not wanting users to know what actually being sent..

However, in this case, it's good for Microsoft to allow users to turn off the feature. And even better if they allow users to inspect what's actually being sent.

Yes, going through the effort here does have diminishing returns at a certain point in time beyond having prettier logs or confidence that no unplanned cert bypass is occurring.

Many years ago, I had to implement a Websense gateway for https inspection and there were comparable challenges faced at the time with a workaround of bypassing the man-in-the-middle inspection for problem sites. It would be nice if it were possible to allow a site exception to avoid slu increase, which would then function the same, but allow for you to more closely monitor for slu increase.

In my case, I cleared data on Skype for Business and it stopped, of course until I reconfigured and the enforced logging (Intune configured) started them up again. At least I know confidently where it's coming from.

@kvic In reading through Page 86 again, am I correct in thinking that if I follow:

If that still not possible in your adblock script, the fallback approach for ASUSWRT-merlin users will be

• whitelist it in adblock script
• manually add entries to /jffs/configs/hosts.add, e.g.
  • 0.0.0.0 t.appflyer.com
  • 0.0.0.0 x.flyme.com

The logs will clear up for sites I am fine with occurring without breaking them from functioning? What I mean is that they will still be able to connect, but won't trigger an slu increase rather than forcing a failure by redirecting to 0.0.0.0 (null). If they increase slu already, is that indicative of a connection failure?

 

[kvic]

Yes, going through the effort here does have diminishing returns at a certain point in time beyond having prettier logs or confidence that no unplanned cert bypass is occurring.

That's a valid point and the methods I mentioned above and you quoted below is the current solution.

However, people could see when the list grows significantly big: 1) it defeats the original purpose of sending advert/tracker traffic to pixelserv-tls for speed up browsing. 2) the list itself becomes clunky to be manageable for various reasons.

Perhaps we can have a new feature in a future version of pixelserv-tls to have better suppression control or privacy breach warning mechanism.

People are invited to think about it and don't hesitate to suggest ideas.

In reading through Page 86 again, am I correct in thinking that if I follow:
The logs will clear up for sites I am fine with occurring without breaking them from functioning?

The logs will clear since the traffic is redirected to 0.0.0.0 in the simple setup. slu won't be increased. If they're increased, it indicates new connection failure and possibly 'dubious' requests.

What I mean is that they will still be able to connect, but won't trigger an slu increase rather than forcing a failure by redirecting to 0.0.0.0 (null).

A slightly more elaborate setup that retains all benefits of pixelserv-tls is to have two instances e.g. on the same router.

Say primary instance on 192.168.1.11 for all requests. A secondary instance on 192.168.1.22 for filtered advert/tracker servers.

Now you could use the primary instance for monitoring everything including slu. The only change is replace 0.0.0.0 with 192.168.1.22 in /jffs/configs/hosts.add.

This is perhaps a reasonably good alternative until we have something better. With the new openssl lib, two instances of pixelserv-tls consumes little memory. And they won't have conflict at all in sharing CERT_PATH and certs inside.

 

[jrmwvu04]

Just as an FYI....with the debian patch applied on my fork, I don't see any difference in memory use (running one VPN Server, one VPN Client, NOT running ABSolution/pixelserv)

I definitely do. It hasn’t passed 40MB yet and normally it used to boot in the low 40s and at this point it would be into at least the high 40s. Not exactly groundbreaking. But definitely different.

 

[john9527]

I definitely do. It hasn’t passed 40MB yet and normally it used to boot in the low 40s and at this point it would be into at least the high 40s. Not exactly groundbreaking. But definitely different.

Good to hear. As I said, I don't run pixelserv so my observation was more 'it didn't break anything' on a stock system. Now, if you validate it works as expected for pixelserv installs it will complete the results.

 

[kvic]

Now, if you validate it works as expected for pixelserv installs it will complete the results.

Just FYI...pixelserv-tls does not make use of FW's openssl library.

As a matter of fact Entware doesn't care about FW much other than the kernel..

 

[john9527]

Just FYI...pixelserv-tls does not make use of FW's openssl library.

I guess I misunderstood then...So you must install the modified openssl from Entware? @RMerlin and I patching the FW openssl won't matter (at least as concerns pixelsrv)?

 

[kvic]

I guess I misunderstood then...So you must install the modified openssl from Entware?

Sounds like you're not a Entware user or perhaps use it little. Let's not digress into that.

Your change has zero impact on pixelserv-tls. Given that said, the flag be good for your FW users in general. I'm not convincing you nor discouraging to turn on the flag.

 

[john9527]

Sounds like you're not a Entware user or perhaps use it little. Let's not digress into that.

Your change has zero impact on pixelserv-tls. Given that said, the flag be good for your FW users in general. I'm not convincing you nor discouraging to turn on the flag.

Silly me....I didn't realize you installed libopenssl as a dependancy....
No need to be condescending in your response.

 

[kvic]

No need to be condescending in your response.

My apology. No offence intended. Just want to be accurate in terms of technical communication.

 

[kvic]

Silly me....I didn't realize you installed libopenssl as a dependancy

It actually would be good to take this chance to elaborate a bit on Entware to readers of this thread.

While OpenWRT is a modular, organised and fully blown FW for routers and alike, Entware is almost everything from OpenWRT minus devices drivers and kernel plus extra packages like pixelserv-tls not available on OpenWRT yet.

People shall consider Entware as a run-time environment. It contains lots of packages (and more functional ones) as well as fundamental packages that include C library, pthread libraries and etc.

Entware depends on very little on the underlying FW. The critical piece of dependency is the kernel. Hence, people will see 3.x kernel for 64-bit Entware (as well as 32-bit ARMv7). In addition, you'll see 32-bit ARMv7 Entware for 2.6.x kernels. There is a reason for such alignment. 'cos kernel is the critical piece to let Entware run.

Other than that Entware is pretty much self-contained. Hence, a clean and diverse development environment too. With FWs such as ASUSWRT-Merlin that provides a friendly environment to install Entware, people shall take it as a win-win situation. So that each side can focus on their own and do its best.

 

[M@rco]

Latest 2.1.0-rc.4 with updated libopenssl:

While memory usage went down tremendously, it seems like tav is being higher on average (anywhere between 50ms and 150ms) than with previous rc's and 'default' libpenssl (0ms - 25ms). Is that a coincidence or related?

 

[kvic]

Latest 2.1.0-rc.4 with updated libopenssl:

While memory usage went down tremendously, it seems like tav is being higher on average (anywhere between 50ms and 150ms) than with previous rc's and 'default' libpenssl (0ms - 25ms). Is that a coincidence or related?

It should be just a coincidence because tav depends on the websites you browse in the past few hours. Let's continue to have an eye on it, and thanks for raising up this concern.

Btw, if your issue persist, pls check if pixelserv-tls has problem reading your CERT_PATH i.e. /opt/var/cache/pixelserv.

 

[jrmwvu04]

It should be just a coincidence because tav depends on the websites you browse in the past few hours. Let's continue to have an eye on it, and thanks for raising up this concern.

Btw, if your issue persist, pls check if pixelserv-tls has problem reading your CERT_PATH i.e. /opt/var/cache/pixelserv.

My tav are higher as well with rc4. Nothing crazy but up into 10-20 ranges as I check periodically. Was often below 10 in recent builds.

 

[kvic]

A way to test tav

A stripped down version of test in #1618. Set to "-O 5" and restart pixelserv-tls.

Browse website_1, wait for 6 seconds, browse webiste_2, wait for 6s and repeat.

website 1 & 2 could be foxnews and cnn.com. This shall provide a worst scenario tav (for a potential issue I could think of).

I'm also going to do some comparison between rc.3 and rc.4 (both using the new lib) later today.

 

[DonnyJohnny]

Sounds like you're not a Entware user or perhaps use it little. Let's not digress into that.

Your change has zero impact on pixelserv-tls. Given that said, the flag be good for your FW users in general. I'm not convincing you nor discouraging to turn on the flag.

A bit off topic, how would then the new lib affect the rest of the usage of OpenSSL like the OpenVPN using it and GUI in https mode. Etc?
I Google around and see no security issue. Not sure about performance but the memory usage for all my installed program like openvpn, dnscrypt-proxy and pixelserv seems to be lower than before.
Don’t know if this is just a coincidence.

 

[quant88]

With 2.1.0-rc.4 and updated libopenssl and -c 300.
After 19+hrs with AC88U, MEM%=0.9

71275 uts, 2 log, 13 kcc, 72 kmx, 1.51 kvg, 184 krq, 33851 req, 1038 avg, 39708 rmx, 37 tav, 5815 tmx, 24734 slh, 20 slm, 0 sle, 872 slc, 7146 slu, 579 uca, 4428 uce, 131 sct, 27900 sch, 56 scm, 0 scp, 17 sst, 6385 ssh, 137 ssm, 0 ssp, 2877 nfe, 10 gif, 6 ico, 110 txt, 1 jpg, 1 png, 0 swf, 8 sta, 0 stt, 14732 ufe, 0 opt, 7858 pst, 1 hed, 103 rdr, 0 nou, 0 pth, 0 204, 56 bad, 48 tmo, 874 cls, 0 cly, 0 clt, 0 err

 

[kvic]

A bit off topic, how would then the new lib affect the rest of the usage of OpenSSL like the OpenVPN using it and GUI in https mode. Etc?
I Google around and see no security issue. Not sure about performance but the memory usage for all my installed program like openvpn, dnscrypt-proxy and pixelserv seems to be lower than before.
Don’t know if this is just a coincidence.

If I understand your situation correctly, what you saw is just a coincidence.

FW has its own openvpn, OpenSSL and etc that do not use the entawar lib.

In order to take the benefit, FW has to rebuild the OpenSSL lib it uses and enables this flag. Hence, the two FW gents are looking into it.

I bet the most demanding ssl task on forum members routers atm is pixelserv-tls. Hence effect might not be as dramatic as you saw on pixelserv-tls.

Nevertheless, the flag shall be enabled IMO. Less code. Use less RAM.

 

[punkinduster]

Was there something in my post that caused this?

(@punkinduster, there was a reply to you pending moderator action)

 

[kvic]

Was there something in my post that caused this?

(@punkinduster, there was a reply to you pending moderator action)

I can see my reply to you. This thread is getting busier.. easy to lose track.. even for myself but after v2.1 release it shall go quieter I believe.

A few subsequent discussions also apply to your situation.

I'm interested in hearing back from people trying two instances of pixelserv-tls (either for load balancing or two IP approach for filtering known "unknown cert" servers).