pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[kvic]

No worries. I appreciate your hard work. Ill test it now.

Pls wait. I figured it won't work. Uploading a new version...

EDIT:

Uploaded pixelserv-tls.Kl-test4.Entware-ng.mipsel.3.zip
Please try this version. Thanks

 

[tom-]

Pls wait. I figured it won't work. Uploading a new version...

EDIT:

Uploaded pixelserv-tls.Kl-test4.Entware-ng.mipsel.3.zip
Please try this version. Thanks

As suspected the second one did fail I'm testing the 3rd one now.
EDIT: 3rd one started and works. Serv stats page loads and all.

 

[tom-]

Does it matter any that the cert I use is 4069 SHA-512? I haven't seen it struggle in generating the certs for the ad domains.

 

[kvic]

Does it matter any that the cert I use is 4069 SHA-512? I haven't seen it struggle in generating the certs for the ad domains.

It shall not, given enough computing power.

Note that 4096 bit key / 512 bit SHA consumes more computing resources (and battery) than a 1024 bit key / 256 bit SHA on both your routers and client devices.

I recommend 1024 bit key / 256 bit SHA as it's the minimum I found today's browsers are happy with.

Stronger encryption provides no additional functionality for our purpose in using pixelserv-tls.

 

[tom-]

It shall not, given enough computing power.

Note that 4096 bit key / 512 bit SHA consumes more computing resources (and battery) than a 1024 bit key / 256 bit SHA on both your routers and client devices.

I recommend 1024 bit key / 256 bit SHA as it's the minimum I found today's browsers are happy with.

Stronger encryption provides no additional functionality for our purpose in using pixelserv-tls.

Okay, I've switched to that then.

 

[kvic]

This is one real reason why people want to consider using pixelserv-tls.

TLDR: stats for geeks but speed for everyone.

 

[tom-]

@kvic
I have some good news with the KL test 4 mipsel 3 you compiled.. No crashes and no bads so far. https://i.imgur.com/zZUZjuS.png

 

[thelonelycoder]

This is one real reason why people want to consider using pixelserv-tls.

TLDR: stats for geeks but speed for everyone.

Thanks for the comparison. I would have guessed that 0.0.0.0 and pixelserv-tls about compare with the times.
Compared to 127.0.0.1 pixelserv-tls would look even better.

I happen to notice this interesting bit you posted about your router:
DNS adblocker with pixelserv-tls and cascading hash tables in Dnsmasq
This sounds like a performance improvement. Could you share how this is done?

 

[kvic]

A new test version is available (KL-test5).

More robust and efficient. Also has enhanced logging for privacy check.

Pls check https://kazoo.ga/pixelserv-tls/ for details.

@thelonelycoder fyi. Turned out I lied when you asked last time if there will be any new CLI options. A new one "-T num" was added in this test.

 

[jrmwvu04]

Changing to test5. Ran test4 for the better part of a few days.

 

[jrmwvu04]

One quick question..

Changed 'select_timeout' default to 1s

I assumed this was the setting defined by the -o flag. I removed my custom -o 1 setting from test5 and the 'tmx' counter registered a 10005 ms reading. Replacing the flag fixed it. If that's what was supposed to change from 10s to 1s, it seems not to work all the time. Will remove the flag and let it run to see if it comes back, see if I can repro, etc. If I'm off on this please correct me.

edit: yeah, dailymail drives it well above 1s

 

[kvic]

I happen to notice this interesting bit you posted about your router:
DNS adblocker with pixelserv-tls and cascading hash tables in Dnsmasq
This sounds like a performance improvement. Could you share how this is done?

Let's make it a bit more fun..

People who are using pixelserv-tls come out and stand up. You can either reply to this thread to claim you're a user or if too lazy simply like post #608. By the coming Christmas eve, if there are 100 or more pixelserv-tls users, I promise to tell the community on Boxing day.

 

[kvic]

@jrmwvu04 thank you.

A few changes like max allowed threads and fix to stale 'kcc' in KL-test5 are direct results of your earlier feedback. Upon closer code inspection, I figured the logics that caused 'kcc' didn't drop below high values.

One quick question..

I assumed this was the setting defined by the -o flag. I removed my custom -o 1 setting from test5 and the 'tmx' counter registered a 10005 ms reading. Replacing the flag fixed it. If that's what was supposed to change from 10s to 1s, it seems not to work all the time.

I'm afraid your observation is just a coincidence.

'-o' flag is select_timeout and its default is changed to 1s in KL-test5.

However, at the same time KL-test5 doubled the maximum allowed wait time (5s to 10s) for receiving POST message (which could be quite huge). So I believe it just happened that there were one or more POSTs in your first tests which hit 10s.

Let's see..we can revert back to 5s for waiting POST in next test.

 

[Butterfly Bones]

Let's make it a bit more fun..

People who are using pixelserv-tls come out and stand up. You can either reply to this thread to claim you're a user or if too lazy simply like post #608. By the coming Christmas eve, if there are 100 or more pixelserv-tls users, I promise to tell the community on Boxing day.

(raises hand)
pixelserv-tls user

 

[kvic]

Anyone is building pixelserv-tls for Edgerouter-X aka mipsel Debian? Would be great to hear from you..I tried to build but got stuck in the last 1% and I'm running short of time this weekend.

I believe pixelserv-tls would be very happy with 4 cores on ER-X versus 2 cores on ARM Cortext A9.

AMD64 users with 'many cores' shall see better speed than what I got with pixelserv-tls running on RT-AC56U in pixelserv-tls : More is Less.

Methodology is documented there. Encourage people to run your own tests. Would love to hear surprises, good or bad.

 

[kvic]

Changing to test5. Ran test4 for the better part of a few days.

Looking closer at the above stats collected over two days, I found a few interesting tidbits which are hard to see in my usage patterns.

1. average process time, tav is 12ms. That's superb. Expect test KL's shall see increasingly lower process time with LAN clients than earlier test versions, and version Kk.

2. average request per thread, kvg, is near 4. Interestingly high.

3. average request size, avg, about 1kB. Mine is around 700bytes.

4. Reasonable largest request size, rmx, about 20kB. Mine is close to 200kB because of devices by Chinese vendors. They love and attempt to send tonnes of data back to their headquarters in my case.

5. HTTP OPTIONS and HEAD requests aren't as rare as people thought. Here we have 12 opt and 6 hed.

 

[tom-]

I'm getting these errors now with certs generated by pixelserv. https://imgur.com/a/PhRfT This problem goes away after I add the pixelserv cert to the browser. I discovered this after removing the old cert and forgot to add the new. I thought before it didn't matter too much if the cert was added to the devices.

EDIT: Also this strange error: https://imgur.com/a/Dj1oE
Only saw that once in the logs. Logs get rotated every 45 mins, so there could have been more. That's the only one I seen. Just briefly checking the logs.
I'm using the KL test 5 mipsel and logging on 4.
EDIT 2: Just noticed another different url all together.: pixelserv-tls[13805]: [ÿþ<]
https://i.imgur.com/OTKNmHC.png

 

[kvic]

I'm getting these errors now with certs generated by pixelserv. https://imgur.com/a/PhRfT This problem goes away after I add the pixelserv cert to the browser. I discovered this after removing the old cert and forgot to add the new. I thought before it didn't matter too much if the cert was added to the devices.

The HSTS mentioned in Firefox error message was an experimental feature added since version KK. It forces supported browsers (almost all recent builds of major browsers) to switch to use HTTPS even if a user enters http://xyz...

However, your error seems coming from not having imported CA cert into the browser? Individual certificates are not required and shall not be added to browsers.

EDIT: Also this strange error: https://imgur.com/a/Dj1oE
I'm using the KL test 5 mipsel and logging on 4.

This is "normal". The data in [] is the HTTP POST content captured by pixelserv-tls at INFO (4) level.

However, I also found some websites claim themselves sending text i.e. Base64/UUencoded POST content but rather sending binary data. Hence, the strange characters when displayed as text.

If this is common, a better thing will be for a future test version to display such data as hexadecimal.

 

[tom-]

The HSTS mentioned in Firefox error message was an experimental feature added since version KK. It forces supported browsers (almost all recent builds of major browsers) to switch to use HTTPS even if a user enters http://xyz...

However, your error seems coming from not having imported CA cert into the browser? Individual certificates are not required and shall not be added to browsers.



This is "normal". The data in [] is the HTTP POST content captured by pixelserv-tls at INFO (4) level.

However, I also found some websites claim themselves sending text i.e. Base64/UUencoded POST content but rather sending binary data. Hence, the strange characters when displayed as text.

If this is common, a better thing will be for a future test version to display such data as hexadecimal.

Only cert I was adding to the browser was the one we create for pixelserv to generate the other certs. I'm on linux. Adding it to the browser is just a simple solution than adding it to the system. And not all machines on my network I have access to to add the cert to the system. Like the windows pc on my network. So, maybe something can be done in the future so, certs don't have to be imported to each device.

And good to know the random characters is normal. I never had it on log level 4. And I wasn't too sure what info it was going to show since I never used it before.

 

[kvic]

Only cert I was adding to the browser was the one we create for pixelserv to generate the other certs. I'm on linux. Adding it to the browser is just a simple solution than adding it to the system.

Adding the CA certificate into Firefox is right. If I recall correctly, FF ignores system CA bundles anyway...So you already did. I tried to with Firefox 58 but cannot reproduce the error.

And not all machines on my network I have access to to add the cert to the system. Like the windows pc on my network.

That's also right. Not all clients are required to import the CA certificate. Clients without it will simply be rejected by pixelserv-tls, similar to hitting the wall like 0.0.0.0 but continue to function. Shall not be noticeable from end-users.

And good to know the random characters is normal. I never had it on log level 4. And I wasn't too sure what info it was going to show since I never used it before.

Logging POST content for privacy inspection is a new feature in KL-test 5.