1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.
Dismiss Notice

Welcome To SNBForums

SNBForums is a community for anyone who wants to learn about or discuss the latest in wireless routers, network storage and the ins and outs of building and maintaining a small network.

If you'd like to post a question, simply register and have at it!

While you're at it, please check out SmallNetBuilder for product reviews and our famous Router Charts, Ranker and plenty more!

pixelserv - A Better One-pixel Webserver for Adblock

Discussion in 'Asuswrt-Merlin' started by kvic, Jul 28, 2015.

  1. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,444
    Location:
    22.4399N 114.2222E
    My rc.4 still going strong. A few pixelserv ninjas have been working hard in the past few days to reproduce the issue.

    We already have a much better understanding what’s the issue. A fix might not be possible as it’s related to very new feature in tls 1.3. I still have to look into it. A workaround should be easy.

    For ppl with access to Discord troubleshooting steps and full discussions are there.

    I personally caught a cold. Hence delayed the update in this thread.

    Luckily we didn’t have to go the route of firing up a PC. It takes longer time and a few more hassle but we got there on Asus routers.
     
    [email protected] likes this.
  2. Thanks for the update @kvic. I hope you feel better soon!
     
    kvic likes this.
  3. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,444
    Location:
    22.4399N 114.2222E
    You may try to run the latest build from Discord if you can/want. It has improvement in handling 0rtt that possibly has caused a hung process.

    Or wait for rc.5 that should be available in a few days time if tests go well.
     
  4. DonnyJohnny

    DonnyJohnny Very Senior Member

    Joined:
    Dec 17, 2017
    Messages:
    695
    I don’t have crush. Can I use that build too?

    Just use the wget in the discord? No need the full command that link with “_binfavor=static sh -c” ?
     
  5. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,444
    Location:
    22.4399N 114.2222E
    The build on Discord is only for ARMv7 routers, and no fancy script to install the binary. Pretty raw..so people should know what you're doing. You may go ahead or simply wait for rc.5. lol
     
  6. DonnyJohnny

    DonnyJohnny Very Senior Member

    Joined:
    Dec 17, 2017
    Messages:
    695
    Lol.. done...
    Code:
    pixelserv-tls[5911]: pixelserv-tls 2.2.0-rc.4 (compiled: Sep 28 2018 03:10:36 flags: tls1_3) options: 192.168.2.3 -c 400
    Need stop diversion first before that -wget.
    I just want to be at the edge of technology. Don’t know what I doing. Just have fun. Hahaha..
     
    Last edited: Sep 28, 2018
    [email protected] likes this.
  7. regae

    regae New Around Here

    Joined:
    Jul 20, 2015
    Messages:
    9
    when is the invitation open again? i'd like to try the non-released build
     
  8. joe scian

    joe scian Senior Member

    Joined:
    Apr 22, 2018
    Messages:
    286
    Thanks Kvic - am trying latest V3 pixelserv-tls 2.2.0-rc.4 (compiled: Sep 28 2018 06:32:17 flags: tls1_3).

    Looks great so far - been giving it a hiding - holding up just fine

    I am getting a few of these though
    Sep 29 12:08:59 pixelserv-tls[32308]: read_tls_early_data timeout
    Sep 29 12:09:08 pixelserv-tls[32308]: read_tls_early_data timeout
    Sep 29 12:09:16 pixelserv-tls[32308]: read_tls_early_data timeout
    Sep 29 12:09:17 pixelserv-tls[32308]: read_tls_early_data timeout
    Sep 29 12:09:30 pixelserv-tls[32308]: read_tls_early_data timeout

    pixelserv-tls 2.2.0-rc.4 (compiled: Sep 28 2018 06:32:17 flags: tls1_3) options: 192.168.2.3


    uts 0d 01:44 process uptime
    log 1 critical (0) error (1) warning (2) notice (3) info (4) debug (5)
    kcc 24 number of active service threads
    kmx 39 maximum number of service threads
    kvg 2.61 average number of requests per service thread
    krq 169 max number of requests by one service thread
    req 3581 total # of requests (HTTP, HTTPS, success, failure etc)
    avg 1166 bytes average size of requests
    rmx 12304 bytes largest size of request(s)
    tav 10 ms average processing time (per request)
    tmx 10331 ms longest processing time (per request)
    slh 1919 # of accepted HTTPS requests
    slm 30 # of rejected HTTPS requests (missing certificate)
    sle 0 # of rejected HTTPS requests (certificate available but bad)
    slc 827 # of dropped HTTPS requests (client disconnect without sending any request)
    slu 72 # of dropped HTTPS requests (other TLS handshake errors)
    uca 0 slu break-down: # of unknown CA reported by clients
    uce 0 slu break-down: # of unknown cert reported by clients
    ush 64 slu break-down: # of shutdown by clients after ServerHello
    sct 50 cert cache: # of certs in cache
    sch 951 cert cache: # of reuses of cached certs
    scm 48 cert cache: # of misses to find a cert in cache
    scp 35 cert cache: # of purges to give room for a new cert
    sst 1410 sess cache: # of cached TLS sessions (for older non-RFC5077 clients)
    ssh 29 sess cache: # of reuses of cached TLS sessions
    ssm 15 sess cache: # of misses to find a TLS session in cache
    ssp 0 sess cache: # of purges to give room for a new TLS session
     
    Last edited: Sep 28, 2018
  9. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,444
    Location:
    22.4399N 114.2222E
    Thanks for your interest first of all!

    We don’t usually do new binaries there. This is probably the first time because it just happens there is an issue and ppl brought up there.

    Also there is no secret nor valuable info there. Rather it’s a more comforting place to exchange ideas e.g. no censorship no freaking buzzwords to prevent u from posting etc. it’s still a public place and everyone should treat it so.

    Some people might like to be there. Quite many never speak up and my evil plan is to kick them out perhaps someday. Lol.

    I would encourage u to speak up more often here. People will know u a bit more and then fit in a comforting place.

    Btw rc.5 should be available in a day or two. Real soon.
     
    jsbeddow likes this.
  10. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,444
    Location:
    22.4399N 114.2222E
    Recently I was made aware of two efforts that is somewhat related to pixelserv-tls.

    1) an OPNsense user finally pops up to create an adblocker for OPNsense.

    It’s still at the early stage. The goal is to beat pfBlockerNG. Another user is already proposing to integrate pixelserv-tls. If things play out, then should be good news.

    Life for pfSense SOHO users in longer term is a bit cloudy as the company behind will focus on a new platform. OPNsense is something to have eye on.

    The relevant thread is here: https://forum.opnsense.org/index.php?topic=9523.0

    2) pixelserv-tls receives some attention in pi-hole world. A new gent is requesting a better interoperability with pixelserv-tls. Pi Hole users should watch this request and upvote it if it’s in your interest.

    https://discourse.pi-hole.net/t/support-dropping-esni-records-for-blocked-domains/13250
     
    Xentrk likes this.
  11. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,444
    Location:
    22.4399N 114.2222E
    Original rc.4 into its 7 days. Tonight hopefully we can release rc.5 with improved compatibility in operations.

    Code:
    pixelserv-tls 2.2.0-rc.4 (compiled: Sep 23 2018 19:12:30 flags: tls1_3) options: 192.168.1.3 -A 344 -l 2 -c 350
    
    uts 6d 16:57 process uptime
    log 2 critical (0) error (1) warning (2) notice (3) info (4) debug (5)
    kcc 1 number of active service threads
    kmx 116 maximum number of service threads
    kvg 2.44 average number of requests per service thread
    krq 3286 max number of requests by one service thread
    req 69379 total # of requests (HTTP, HTTPS, success, failure etc)
    avg 1546 bytes average size of requests
    rmx 149723 bytes largest size of request(s)
    tav 49 ms average processing time (per request)
    tmx 14073 ms longest processing time (per request)
    slh 50227 # of accepted HTTPS requests
    slm 33 # of rejected HTTPS requests (missing certificate)
    sle 0 # of rejected HTTPS requests (certificate available but bad)
    slc 5367 # of dropped HTTPS requests (client disconnect without sending any request)
    slu 5972 # of dropped HTTPS requests (other TLS handshake errors)
    uca 0 slu break-down: # of unknown CA reported by clients
    uce 0 slu break-down: # of unknown cert reported by clients
    ush 2097 slu break-down: # of shutdown by clients after ServerHello
    
     
  12. Xentrk

    Xentrk Part of the Furniture

    Joined:
    Jul 21, 2016
    Messages:
    2,361
    Location:
    The Land of Smiles
    I have appeared on the pi-hole discourse forum a few times trying my best to persuade the pi-hole devs to include pixelserv-tls. They mentioned the MTM concern the first time. Thanks for sharing the update!
     
  13. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,444
    Location:
    22.4399N 114.2222E
    I believe your persistence worked out, particularly the last round. So perhaps you should be proud of the difference you made.

    I also had the gut feeling ppl picked up a thing or two from my blog or some of the discussion we did on this forum. Devs surely carry a friendlier tone than before. Misconception in users may need more time to clear out.

    Btw if u want a project with profound impact, perhaps you should consider participating in the opnblocker for opnsense.
     
    Clark Griswald and Xentrk like this.
  14. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,444
    Location:
    22.4399N 114.2222E
    2.2.0-rc.5 is available

    We solved a difficult problem. For changes, pls see kazoo.ga/pixelserv-tls/

    The debug process and credits

    As some of you may know, rc.4 gets into a "hung" state for some people. To work out this issue, we get many helps from a few pixelserv-tls ninjas.

    In retrospect, pixelserv-tls never actually hangs. It's like as @elorimer described that the response returned after 26s of waiting. That was corroborated in @[email protected]'s description in his case that came up later. For unknown reasons, when the issue happens and it's inside OpenSSL, it takes a varying amount of time to get itself out the stuck state but it did get out eventually. While pixelserv-tls gets in such a stuck state, a flood of requests might actually crash the process though very rare. I recall a crash only happened once as per @Protik's description.

    We should thank all the mentioned names in their contribution to get a better understanding of the problem. @Protik in particular was the first one reporting the issue and ever since worked enthusiastically in getting to the bottom of this issue. That really kept the momentum going inside the silo.

    I should not forget to mention @quant88 who happened to encounter the issue, responded quickly and was able to swiftly take the first backtraces. If I didn't remember wrongly he was also one of first guys a year ago that helped solving the riddle of a hung pixelserv-tls. The first backtraces really opened up the problem to a new level.

    Luckily we also have @jrmwvu04 who in the middle of the process jumped to the frontline with a 100% reproducible case that was dearly sought after and was left on the table idle for a little while. From there the understanding of the issue and possible fixes were almost all on the plate, and we were able to proceed swiftly to a conclusion.

    Forgot to mention @joe scian who also provided a 100% reproducible case at later stage.

    Thank you.
     
    visortgw, Quoc Huynh, Xentrk and 10 others like this.
  15. Clark Griswald

    Clark Griswald Regular Contributor

    Joined:
    Sep 21, 2015
    Messages:
    197
    Location:
    Northern California, USA
    Thank You to Kvic and everyone that contributes!

    P.S. the update from rc.4 to rc.5 was smooth and easy.
     
    Twiglets and kvic like this.
  16. Asad Ali

    Asad Ali Senior Member

    Joined:
    May 25, 2017
    Messages:
    452
    Location:
    Pakistan
    Where's my name?? I provided moral support since I'm on 86U
     
    Twiglets likes this.
  17. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,444
    Location:
    22.4399N 114.2222E
    Oh [email protected] Ali is the spirit of the silo

    We also have @truglodite for guest shot & brief appearance, and later @DonnyJohnny for a brief of intensive tests.

    Also we have to thank the janitors of the silo..and the beautiful tea & coffee ladies.

    thanks all!
     
  18. SMS786

    SMS786 Regular Contributor

    Joined:
    Nov 29, 2017
    Messages:
    177
    What's the command to install rc5 with TLS 1.3?
     
  19. Twiglets

    Twiglets Regular Contributor

    Joined:
    Aug 15, 2014
    Messages:
    176
    Quoc Huynh and SMS786 like this.
  20. kvic

    kvic Part of the Furniture

    Joined:
    Aug 11, 2014
    Messages:
    2,444
    Location:
    22.4399N 114.2222E
    It's on the release notes. Apparently you never read. LOL
     
    visortgw, hervon and Twiglets like this.