pixelserv pixelserv - A Better One-pixel Webserver for Adblock

[Raphie]

So here is a good 5h of use with certificates on all client devices

pixelserv-tls: v35.HZ12.Kl-test8c compiled: Nov 20 2017 11:07:00 options: 192.168.1.2
24111 uts, 1 log, 2 kcc, 18 kmx, 1.35 kvg, 20 krq, 2952 req, 821 avg, 3769 rmx, 40 tav, 989 tmx, 1436 slh, 93 slm, 0 sle, 233 slc, 127 slu, 458 nfe, 6 gif, 4 ico, 261 txt, 0 jpg, 8 png, 0 swf, 22 sta, 4 stt, 87 ufe, 15 opt, 21 pst, 0 hed, 142 rdr, 0 nou, 0 pth, 0 204, 0 bad, 43 tmo, 207 cls, 696 cly, 0 clt, 0 err

 

[thelonelycoder]

I've made a manpage for pixelserv-tls.

Proof read & feedback welcome as well as comments to other documentation on the wiki page.

Excellent, thanks. Will read it later and also include the link in AB-Solution in the switches setting page.

 

[kvic]

So here is a good 5h of use with certificates on all client devices

Running healthy and it's huge progress from before

 

[Raphie]

Check! thank you guys, I've learned a few things today

 

[tom-]

if ssh into your router, and then

openssl ciphers

What is the full output?

ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:SRP-DSS-AES-256-CBC-SHA:SRP-RSA-AES-256-CBC-SHA:SRP-AES-256-CBC-SHA:DH-DSS-AES256-GCM-SHA384:DHE-DSS-AES256-GCM-SHA384:DH-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA256:DH-RSA-AES256-SHA256:DH-DSS-AES256-SHA256:DHE-RSA-AES256-SHA:DHE-DSS-AES256-SHA:DH-RSA-AES256-SHA:DH-DSS-AES256-SHA:ECDH-RSA-AES256-GCM-SHA384:ECDH-ECDSA-AES256-GCM-SHA384:ECDH-RSA-AES256-SHA384:ECDH-ECDSA-AES256-SHA384:ECDH-RSA-AES256-SHA:ECDH-ECDSA-AES256-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA:PSK-AES256-CBC-SHA:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:SRP-DSS-AES-128-CBC-SHA:SRP-RSA-AES-128-CBC-SHA:SRP-AES-128-CBC-SHA:DH-DSS-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:DH-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES128-SHA256:DHE-DSS-AES128-SHA256:DH-RSA-AES128-SHA256:DH-DSS-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA:DH-RSA-AES128-SHA:DH-DSS-AES128-SHA:ECDH-RSA-AES128-GCM-SHA256:ECDH-ECDSA-AES128-GCM-SHA256:ECDH-RSA-AES128-SHA256:ECDH-ECDSA-AES128-SHA256:ECDH-RSA-AES128-SHA:ECDH-ECDSA-AES128-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:PSK-AES128-CBC-SHA:ECDHE-RSA-RC4-SHA:ECDHE-ECDSA-RC4-SHA:ECDH-RSA-RC4-SHA:ECDH-ECDSA-RC4-SHA:RC4-SHA:RC4-MD5:PSK-RC4-SHA:ECDHE-RSA-DES-CBC3-SHA:ECDHE-ECDSA-DES-CBC3-SHA:SRP-DSS-3DES-EDE-CBC-SHA:SRP-RSA-3DES-EDE-CBC-SHA:SRP-3DES-EDE-CBC-SHA:EDH-RSA-DES-CBC3-SHA:EDH-DSS-DES-CBC3-SHA:DH-RSA-DES-CBC3-SHA:DH-DSS-DES-CBC3-SHA:ECDH-RSA-DES-CBC3-SHA:ECDH-ECDSA-DES-CBC3-SHA:DES-CBC3-SHA:PSK-3DES-EDE-CBC-SHA

I'm marking this issue as resolved as I think its something on my end that has got corrupt, because wget-ssl was also having strange issues. So, I am removing everything and doing a fresh entware-ng install and everything else.

 

[elorimer]

A reminder: AB-solution sets the owner to "nobody" whenever you restart pixelserv through it.

I'm not entirely sure about that. For most of the .kl betas I followed the method you posted earlier (stop pixelserv through ab-s, rename the existing .kl beta, copy over the new unzipped beta, change name and properties, restart pixelserv through ab-s). For kl8c I used the update script, and somewhere in that ended up with the owner not being nobody.

I manually changed the owner of the directory to nobody, and things started working. Then I used the update script to go to 8d, and the directory is still nobody. So somewhere along the line I introduced this error.

 

[Protik]

I think I may have found a bug. There are times when pixelserv test8d becomes unresponsive. I am not so sure about what causes it. This happened couple of times but I did not pay much attention. Today this happened when I was streaming using YouTube.

By unresponsive I mean the statistics page shows loading but never loads any information. As the pixelserv service becomes unresponsive, no website completes loading. The browser loads partial website the just loading and loading. It seems streaming makes pixelserv so busy that it can't serve to other requests. When I stopped the streaming everything went back to normal momentarily.

I did a check on the pixelserv-tls service and found,

# /opt/etc/init.d/S80pixelserv-tls check
 Checking pixelserv-tls (AB-Solution)...              alive.

I use pixelserv with Ab-Solution. When pixelserv is unresponsive, Ab-Solution fails to load and gets stuck while doing installation check.

# ab-solution
    _   ___     ___      _      _   _
   /_\ | _ )___/ __| ___| |_  _| |_(_)___ _ _
  / _ \| _ \___\__ \/ _ \ | || |  _| / _ \ ' \
 /_/ \_\___/   |___/\___/_|\_,_|\__|_\___/_||_|

 Welcome
 This is AB-Solution 3.9.3.3

 looking for an installation
 found AB-Solution on /tmp/mnt/pdas001
 checking installation state

Is there any way to identify what's going wrong?

 

[jrmwvu04]

Is there any way to identify what's going wrong?

I had a similar issue many builds ago, resolved by setting up a swap file. I don’t know if it’s back, I haven’t tested and don’t have the free time to do so right now but I’ll check at some point if you haven’t figured it out yet.

 

[kvic]

I'm marking this issue as resolved as I think its something on my end that has got corrupt, because wget-ssl was also having strange issues. So, I am removing everything and doing a fresh entware-ng install and everything else.

I've been figuring out what could have caused it but get no where. I thought it might be cipher related but from your output earlier it wasn't. Glad to hear you got it solved

 

[kvic]

Is there any way to identify what's going wrong?

When you have time, could you try to reproduce the issue? And then when the issue happens again, ssh into your router, and issue command "killall -SIGUSR1 pixelserv-tls"

Then check your syslog, you shall see something like below:

Nov 24 04:27:15 Phaeo pixelserv-tls[29530]: 381906 uts, 1 log, 0 kcc, 66 kmx, 1.56 kvg, 98 krq, 21303 req, 1327 avg, 126256 rmx, 45 tav, 14429 tmx, 10365 slh, 785 slm, 0 sle, 167 slc, 418 slu, 7854 nfe, 119 gif, 21 ico, 5906 txt, 183 jpg, 22 png, 0 swf, 114 sta, 17 stt, 361 ufe, 0 opt, 3395 pst, 6 hed, 1312 rdr, 0 nou, 2 pth, 0 204, 126 bad, 88 tmo, 548 cls, 0 cly, 0 clt, 1 err

Post yours and let's check.

 

[Raphie]

New log

pixelserv-tls: v35.HZ12.Kl-test8c compiled: Nov 20 2017 11:07:00 options: 192.168.1.2
164857 uts, 1 log, 1 kcc, 18 kmx, 1.63 kvg, 25 krq, 9223 req, 761 avg, 3769 rmx, 84 tav, 87061 tmx, 3879 slh, 150 slm, 0 sle, 1904 slc, 205 slu, 1704 nfe, 9 gif, 4 ico, 506 txt, 0 jpg, 14 png, 0 swf, 38 sta, 11 stt, 241 ufe, 52 opt, 62 pst, 0 hed, 669 rdr, 0 nou, 0 pth, 0 204, 0 bad, 89 tmo, 1877 cls, 1865 cly, 0 clt, 0 err

 

[thelonelycoder]

I think test8d runs really well on my 87U:

pixelserv-tls: v35.HZ12.Kl-test8d compiled: Nov 21 2017 01:09:16 options: 192.168.2.2
173075 uts, 1 log, 5 kcc, 42 kmx, 2.04 kvg, 69 krq, 20999 req, 646 avg, 24120 rmx, 15 tav, 2482 tmx, 4388 slh, 269 slm, 0 sle,
100 slc, 96 slu, 2114 nfe, 77 gif, 4 ico, 1814 txt, 21 jpg, 6 png, 0 swf, 24 sta, 9 stt, 106 ufe, 5 opt, 347 pst, 0 hed, 340 rdr, 
1 nou, 0 pth, 0 204, 0 bad, 9 tmo, 195 cls, 150 cly, 0 clt, 0 err

 

[Protik]

When you have time, could you try to reproduce the issue? And then when the issue happens again, ssh into your router, and issue command "killall -SIGUSR1 pixelserv-tls"

Then check your syslog, you shall see something like below:

Nov 24 04:27:15 Phaeo pixelserv-tls[29530]: 381906 uts, 1 log, 0 kcc, 66 kmx, 1.56 kvg, 98 krq, 21303 req, 1327 avg, 126256 rmx, 45 tav, 14429 tmx, 10365 slh, 785 slm, 0 sle, 167 slc, 418 slu, 7854 nfe, 119 gif, 21 ico, 5906 txt, 183 jpg, 22 png, 0 swf, 114 sta, 17 stt, 361 ufe, 0 opt, 3395 pst, 6 hed, 1312 rdr, 0 nou, 2 pth, 0 204, 126 bad, 88 tmo, 548 cls, 0 cly, 0 clt, 1 err

Post yours and let's check.

Here is the output:

Nov 24 01:17:44 pixelserv-tls[11594]: 186279 uts, 1 log, 1 kcc, 68 kmx, 1.92 kvg, 85 krq, 15106 req, 1058 avg, 69048 rmx, 55 tav, 3711 tmx, 5287 slh, 241 slm, 0 sle, 6584 slc, 99 slu, 1721 nfe, 567 gif, 42 ico, 1592 txt, 0 jpg, 9 png, 0 swf, 106 sta, 24 stt, 235 ufe, 0 opt, 1916 pst, 0 hed, 323 rdr, 3 nou, 1 pth, 0 204, 6 bad, 791 tmo, 6156 cls, 43 cly, 0 clt, 0 err

Edit: I can confirm it happens while streaming. When Ab-Solution UI starts, it somehow probes for pixelserv. As Ab-Solution hangs while showing "Checking installation state", the command Ab-solution uses to probe pixelserv could help troubleshoot this.

 

[Protik]

I had a similar issue many builds ago, resolved by setting up a swap file. I don’t know if it’s back, I haven’t tested and don’t have the free time to do so right now but I’ll check at some point if you haven’t figured it out yet.

I already have swap set up. So it may be something else.

# free
             total       used       free     shared    buffers     cached
Mem:        255708     130844     124864          0       3584      14412
-/+ buffers/cache:     112848     142860
Swap:       262140       2780     259360

 

[kvic]

Here is the output:

Nov 24 01:17:44 pixelserv-tls[11594]: 186279 uts, 1 log, 1 kcc, 68 kmx, 1.92 kvg, 85 krq, 15106 req, 1058 avg, 69048 rmx, 55 tav, 3711 tmx, 5287 slh, 241 slm, 0 sle, 6584 slc, 99 slu, 1721 nfe, 567 gif, 42 ico, 1592 txt, 0 jpg, 9 png, 0 swf, 106 sta, 24 stt, 235 ufe, 0 opt, 1916 pst, 0 hed, 323 rdr, 3 nou, 1 pth, 0 204, 6 bad, 791 tmo, 6156 cls, 43 cly, 0 clt, 0 err

Edit: I can confirm it happens while streaming. When Ab-Solution UI starts, it somehow probes for pixelserv. As Ab-Solution hangs while showing "Checking installation state", the command Ab-solution uses to probe pixelserv could help troubleshoot this.

kmx is the maximum number of threads. 68 is a low number. it means pixelserv-tls was not loaded when the hang happens. On my 56U, over kmx >= 2000, pixelserv-tls and the overall system are still responsive. I think it's could be something else that starved the pixelserv process.

When the "hang" happens, you can also use "top" to monitor what processes are using your CPU most. A colour version called "htop" is available from Entware-ng. Its looks like this:

http://hisham.hm/htop/htop-2.0.png

%CPU shows how much % of CPU time used by a process.

 

[Butterfly Bones]

@Protik @elorimer

I suspect the CA certificate has issue in your cases. Can you try following the instruction in the same tutorial:

1. clean up everything in /opt/var/cache/pixelserv
2. follow the tutorial to generate the CA certificate
3. restart pixelserv-tls (/opt/etc/init.d/S80pixelserv-tls restart)
4. import the new CA cert into a client

I have done all these steps, got my ca.crt and successfully imported into two Android, two Linux computers, now I am trying to do Chromebook (ChromeOS) and no luck. I've found tutorials on this, but when I try to import the crt I get this:

The Private Key for this Client Certificate is missing or invalid.

All the info I can find from Googling for help to import chromebook certificates is that this has to be done from the admin console in business or schools that administer chromebooks.

Anyone have experience with this or can point me in the right direction? My Google Fu is well developed but this one evades me.

edit - still searching this. Do I need the ca.key in the same directory? I am hesitant to experiment with certificates like I am willing to do with most anything else.

 

[kvic]

Anyone have experience with this or can point me in the right direction? My Google Fu is well developed but this one evades me.

edit - still searching this. Do I need the ca.key in the same directory? I am hesitant to experiment with certificates like I am willing to do with most anything else.

Have you tried the steps under "ChromeOS" section of the following link?

https://support.securly.com/hc/en-u...install-the-Securly-SSL-certificate-in-Chrome

Only 'ca.crt' is required to be imported. 'ca.key' is only for use by pixelserv-tls.

I don't have a Chromebook so I can't test. Look at the above screenshots, the certificate management is same as Chrome browser. Under different OS, the underlying system repository used by Chrome/ChromeOS may be different. However, if you follow the Chrome/ChromeOS you shall consistently get through its GUI, and have the CA cert imported into the underlying OS.

 

[kvic]

Like most of you posted above, my latest run lasts more than 5 days so far. Thanks for my 56U didn't crash (>3 days is rare recently).

https://imgur.com/a/eTnnD

Will see if @Protik has update on his issue. Perhaps will issue a final test with minor changes later today. This test build will be equivalent to release (touch wood if no further issues discovered).

 

[Butterfly Bones]

Have you tried the steps under "ChromeOS" section of the following link?

https://support.securly.com/hc/en-u...install-the-Securly-SSL-certificate-in-Chrome

Only 'ca.crt' is required to be imported. 'ca.key' is only for use by pixelserv-tls.

I don't have a Chromebook so I can't test. Look at the above screenshots, the certificate management is same as Chrome browser. Under different OS, the underlying system repository used by Chrome/ChromeOS may be different. However, if you follow the Chrome/ChromeOS you shall consistently get through its GUI, and have the CA cert imported into the underlying OS.

That worked, thank you. I missed using the "Authorities" tab, having "User", "Server" and "Authorities", and "Other". I tried User and that did not work. I'm new enough with networking that I was not clear on exactly what to do, that link really helped.

Also I notice the difference in page loading now on the clients, definitely faster.

 

[kvic]

@Butterfly Bones

Thank for confirming! I'll add the link to the same wiki later. Good to have a new supported client on that page