HELLO_wORLD
Very Senior Member
The forum is blacklisting some strings.
Kamoj Kamoj Add-on Beta testing
- Thread starter kamoj
- Start date
@Voxel,
Please take a look at the code at https://github.com/SVoxel/R7800/blo...77b17e8c3a99/git_home/uhttpd.git/uhttpd-cgi.c
It seems that it is hard-coded for a 3-second timeout after the header is sent, as highlighted in the attached screen snippet.
Would it be possible to fix the code to use the value specified by the "-t seconds" command-line option? Your thoughts?
Thanks as always for your custom firmware.
Attachments
Voxel
Part of the Furniture
Not quite so as I understand. Not always 3 seconds. Endless loop:
Code:
. . .
int content_length = 0;
int header_sent = 0;
int download = 0;
. .
while( 1 )
{
. . .
timeout.tv_sec = (header_sent < 1) ? cl->server->conf->script_timeout : 3;
. . .when the value of variable 'header_sent' (initially it is set to '0') could be set to '1'. So initial timeout is set to "-t seconds". Later could be changed to '3' .
Just to change it to the "-t seconds": it is necessary to trace all the logic of this function and w/o real testing...
Let us do the following, more flexible:
(1) I created the test version of uhttpd:
(2) Additional option "-u seconds" is added to this version.
(3) If no "-u seconds" option is used in command line then just '3' will be used as before.
(4) If you want to change '3' to other value use "-u seconds" option. For example if both "-t 60" and "-u 60" are used it means that you have always '60' as timeout.
NOTE: stop uhttpd before replacement of firmware version od /usr/sbin/uhttpd by version from uhttpd-test.tar.gz
Code:
/etc/init.d/uhttpd stopand you can add your "-u seconds" to /www/cgi-bin/uhttpd.sh :
Code:
. . .
$UHTTPD_BIN -h /www -r ${REALM} -x /cgi-bin -t 40 -u 40 ...
. . .after that
Code:
/etc/init.d/uhttpd startP.S.
Please share your results of testing.
Voxel.
kamoj
Very Senior Member
Thank you all (@HELLO_wORLD , @Voxel and others of course) for incredible support and help in this forum!
Your support means really much to me.
I've been a bit indisposed lately, but will try to be more active now. thank you!
The 3 seconds timeout has caused (and causes) many problems with e.g. the bypass add-on "show list" function
when you have many devices (40+), and I've tried to optimize.
Also as users have noted, when the router is loaded, the add-on fails to present information.
These are all because of these 3 seconds.
As for @n1llam1 original issue, it should never take 3 seconds, so I'll look into that and try to find (and fix) the reason.
(There are many bugs in Netgear WiFi-handling...)
Thank you @n1llam1 for all your time and help with improving the add-on!
For functions like OpenVPN Client that takes very long time (eg to ping many hundreds of servers) I've added code that
responds with "dummy answers" to postpone the timeout for up to 90 seconds.
I've had the idea to one day implement a generic "dummy answers" function to all requests,
but prioritized bug-fixes and new functionality.
Like now I'm implementing a WiFi-supervision with automatic radio/wifi-repair.
(This is because there has been a lot of complaints on Netgear routers last 6 months or so, where radios/wifi has stopped
working using "new" Netgear stock Firmware. Many users even think the HW is broken, but it's not!
And Netgear seems to not care at all.
With this supervision/repair I hope to get more people to use the vastly much better Voxel/Aegis software.).
Your support means really much to me.
I've been a bit indisposed lately, but will try to be more active now. thank you!
The 3 seconds timeout has caused (and causes) many problems with e.g. the bypass add-on "show list" function
when you have many devices (40+), and I've tried to optimize.
Also as users have noted, when the router is loaded, the add-on fails to present information.
These are all because of these 3 seconds.
As for @n1llam1 original issue, it should never take 3 seconds, so I'll look into that and try to find (and fix) the reason.
(There are many bugs in Netgear WiFi-handling...)
Thank you @n1llam1 for all your time and help with improving the add-on!
For functions like OpenVPN Client that takes very long time (eg to ping many hundreds of servers) I've added code that
responds with "dummy answers" to postpone the timeout for up to 90 seconds.
I've had the idea to one day implement a generic "dummy answers" function to all requests,
but prioritized bug-fixes and new functionality.
Like now I'm implementing a WiFi-supervision with automatic radio/wifi-repair.
(This is because there has been a lot of complaints on Netgear routers last 6 months or so, where radios/wifi has stopped
working using "new" Netgear stock Firmware. Many users even think the HW is broken, but it's not!
And Netgear seems to not care at all.
With this supervision/repair I hope to get more people to use the vastly much better Voxel/Aegis software.).
HELLO_wORLD
Very Senior Member
Very interesting!
I will have to understand in which circumstances this 3 seconds timeout is triggered.
I created this test script, placed in /www/bolemo/cgi-bin/test.cgi
When I go to routerlogin.net/bolemo/cgi-bin/test.cgi
1) the page takes 10 seconds to load, and all the expected output appears at once. No timeout.
[EDIT] I precise that this is with default uhttpd, not the modified version by @Voxel .
@n1llam1 : could you test this script and see if it times out for you after 3 seconds?
I will have to understand in which circumstances this 3 seconds timeout is triggered.
I created this test script, placed in /www/bolemo/cgi-bin/test.cgi
Code:
#!/bin/sh
sleep 5
echo 'this text is generated after 5 seconds.'
sleep 5
echo 'and this one after 10!'When I go to routerlogin.net/bolemo/cgi-bin/test.cgi
1) the page takes 10 seconds to load, and all the expected output appears at once. No timeout.
[EDIT] I precise that this is with default uhttpd, not the modified version by @Voxel .
@n1llam1 : could you test this script and see if it times out for you after 3 seconds?
Last edited:
I really appreciate the quick turnaround with a modified version of uhttpd that is more flexible with timeout setting options.
It is working as I hoped it would!
I'm happy to report that by adding the -u 40 command-line option, uhttpd now allows a cgi-bin script to run to completion even if they take more than 3 seconds to complete their work.
For example, when I used the WiFi info show button on the Kamoj Menu->System Information page, the cgi-bin script took 7.95 seconds and the browser receives the response data of 2.3 KB and the information is displayed as expected.
Similarly, the OS Log show button has the cgi-bin script taking 3.51 seconds to complete and the 70.3 KB of response data is received and displayed correctly.
The modified uhttpd with the longer timeout allows the add-on cgi script to run to completion and perform it intended functions. Many thanks.
As a side note, when uhttpd is started up (either the original or the modified version), it generates these lines of output in bold below and they appear to be harmless:
root@R7800:/usr/sbin$ /etc/init.d/uhttpd start
/etc/rc.common: /etc/rc.common: 90: detplc: not found
root@R7800:/usr/sbin$ AUTO FW CHECK: power cycle
week_day == 1
killall: aws-iot: no process killed
HELLO_wORLD
Very Senior Member
Really nice, and offers at least one working solution for @kamoj addon, but...
Why don’t I have this limitation as I mentioned earlier?
Have you tried the test script with untouched uhttpd?
Voxel
Part of the Furniture
Good. I am glad to hear! Really.
So, I am interested in such joint cooperation: you find what could be the reason of an issue. I am able to provide the fix.
Your problem is a bit specific. Therefore this new option now is not included into latest release of my build. But I'll add it ("-u seconds") into my git (next releases). Have to check. E.g. with R9000/R8900.
Yeah... Just ignore all of this. Minor issues from NG/DNI...
Voxel.
I tried the test script above and the default uhttpd is behaving as you described. The entire output is displayed after 10 seconds.
this text is generated after 5 seconds.
and this one after 10!
With the clues in the modified code from Voxel, I think I have been able to narrow down that the 3-second timeout is triggered if the cgi-bin script sends back some headers before completing the rest of its work. To prove that theory, I used the following script and saw it get cut off at 3 seconds without any output coming back.
Code:
#!/bin/sh
echo "Content-type: text/html"
echo ""
sleep 5
echo 'this text is generated after 5 seconds.'
sleep 5
echo 'and this one after 10!'HELLO_wORLD
Very Senior Member
Ah! That is becoming clear.
I think @kamoj is sending the header first, I don’t with aegis web GUI.
So it gives 2 solutions: one requires to not send the header and does not require modified uhttpd.
The other is changing uhttpd.
I personally prefer solution #1, as it does not alter uhttpd and allow possible future support of lighttpd (Orbi, new firmwares...)
kamoj
Very Senior Member
Thank you for tests and suggestions.
I agree it's better not change uhttpd.
BUT, the problem without the header is that the size of the response is limited to 4094 bytes.
You can modify your test scripts to return 4095 bytes, which will fail without header.
#Working:
Code:
#!/bin/sh
echo "Content-type: text/html"
echo ""
printf "%4095s" |tr " " "X"#Not working:
Code:
printf "%4095s" |tr " " "X"So either it's long timeout OR long output.
So long outputs that takes >3 seconds to generate will not work without complicated polling until success.
I don't have solution for this.
Any more tips?
Looking on the web, a response header is required, as indicated at these links below:
The solution for the as-is uhttpd is to avoid any delay between the response header and the rest of the response.
For example, the following test.cgi script works, even though it takes longer than 3 seconds and uhttpd returns an HTTP status 500 because of the size of the response:
Code:
#!/bin/sh
RESP=$(echo "Content-type: text/html"; echo ""; sleep 4; printf "%4095s" |tr " " "X")
echo $RESPThe browser receives the response as (I've susbstituted "..." for the remaining X's):
Code:
The CGI program generated an invalid response:
Content-type: text/html
XXXXXXXXXXXXXXXXXXXXXXX...
Last edited:
HELLO_wORLD
Very Senior Member
@kamoj , like @n1llam1 mentions, the test code you proposed without the header is returning this:
Code:
The CGI program generated an invalid response:
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXWith http code 500.
What I do in aegis, is never send a header, and when it returns error 500, I just ignore it and strips "The CGI program generated an invalid response:"
Here is the relevant code in aegis html:
Code:
if (this.readyState == 4) {
switch (this.status) {
case 500:
xhttpOutput = this.responseText.substring(47);
break;
case 200:
xhttpOutput = this.responseText;
break;
default:
alert('Error loading, code ' + this.status);
xhttpOutput = '';
}kamoj
Very Senior Member
Thank you @n1llam1 and @HELLO_wORLD and @Voxel ! 
I tried a mix of different ideas that seems to be fine.
But I've got a lot more to learn about this!
Have not tried to see what is maximum length of message or delay time yet though.
Is there a difference between GET and POST methods?
I tried a mix of different ideas that seems to be fine.
But I've got a lot more to learn about this!
Have not tried to see what is maximum length of message or delay time yet though.
Is there a difference between GET and POST methods?
Glad to hear it's going well trying out different ideas.Thank you @n1llam1 and @HELLO_wORLD and @Voxel !
I tried a mix of different ideas that seems to be fine.
But I've got a lot more to learn about this!
Have not tried to see what is maximum length of message or delay time yet though.
Is there a difference between GET and POST methods?
There is no observable difference in the response as processed by uhttpd for a GET vs. a POST request.
From the command-line, I tried the following and got the same response back for both GET and POST:
Code:
Regular GET request:
curl http://localhost/bolemo/cgi-bin/test.cgi
POST request:
curl --request POST http://localhost/bolemo/cgi-bin/test.cgiHELLO_wORLD
Very Senior Member
With curl to localhost, I am surprised you can get the output without having 401 error. I just tried and it works apparently to call CGI scripts directly.
I had tried in the past to call curl http://localhost/DEV_device.htm and force the refresh of the internal device list in /tmp/netscan/attach_device but it returns the router’s 401 page. Tried again, same error. Anyway, this is off topic.
I confirm that POST or GET does not change anything except how the CGI script receives its data (either in env $QUERY_STRING for GET or stdin for POST). The output, timeouts, etc... are similar.
And about this:
Code:
...
RESP=$(echo "Content-type: text/html"; echo ""; sleep 4; printf "%4095s" |tr " " "X")
...That’s why I don’t bother with sending headers, since it works fine without, and it avoid having to deal with a 3 seconds timeout and/or the 4094 bytes length limitation, as long as the http 500 code is being processed in the http client (JavaScript ajax), and the first 47 chars (string added by uhttpd) are then ignored.
BTW, not sending headers works also in Orbi’s lighttpd.
Because I decided to not send headers from the beginning, I never encountered the 3 seconds timeout problem (and some process in aegis is past 3 seconds) and was not even aware of its existence until this discussion.
Anyway, I think we made good progress here in understanding more uhttpd and its limitations, and how to overcome them.
Now, if only we could crack this net-cgi thing...
Altering the curl command-line a little bit (adding -i --raw options), we see that uhttpd is generating and inserting its own response headers for status 500, text/plain content type, chunked encoding and "The CGI program generated an invalid response:" in front of the actual response generated by the CGI script. I don't know what the numbers 30 and 1000 represent.
Code:
root@R7800:/$curl -i --raw http://localhost/bolemo/cgi-bin/test.cgi
HTTP/1.1 500 Internal Server Error
Content-Type: text/plain
Transfer-Encoding: chunked
30
The CGI program generated an invalid response:
1000
Content-type: text/html XXXXXXXXXXXXXXXXXXXXXXXXXXXXX...When the test.cgi script is changed to generate only 40 characters instead of 4095, we get the following output. As before, I don't know that the numbers 41 and 0 represent. uhttpd inserted its own headers for status 200, text/plain content type and chunked encoding in front of the actual response generated by the CGI script.
Code:
root@R7800:/$ curl -i --raw http://localhost/bolemo/cgi-bin/test.cgi
HTTP/1.1 200 OK
Content-Type: text/plain
Transfer-Encoding: chunked
41
Content-type: text/html XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
0HELLO_wORLD
Very Senior Member
Great, so uhttpd is taking care of headers anyway
The WiFi-supervision with auto-repair feature would really be great. I am still getting random drops of my 5G band (channel 40) on my R7800 AP. Luckily my wireless devices would still be able connect to my main R7800 router on its 5G band (channel 153). Both R7800s have @Voxel V1.0.2.83SF, @kamoj Add-on V5.4b23. Only the router device has @HELLO_wORLD Aegis 1.7.7 installed as well as Entware.
This time the 5G band dropped on my AP device in less than 2 days of uptime, whereas the main router device is still functioning normally with about the same uptime.
The symptoms observed include:
1. The SSID on my AP's 5G band would disappear from my WiFi scanner apps
2. On the /RST_statistic.htm page, the counters for TxPkts and RxPkts for WLAN a/n/ac would stop increasing (attached screen snippets showing counters stuck at 48311074 and 36102555, even after 45 minutes have passed).
3. Eventually, the Kamoj Menu Router Information page would show no devices connected (indicated by empty list for RSSI)
Code:
5G WiFi0 (ath0) ESSID:"5GBAND" , CH:36+40(p)+44+48, Frequency:5.2 GHz Rate:1.7333 Gb/s, Tx-Power (100%):29 dBm (794 mW), RSSI:Normally I would reboot the affected device to get it back to normal.
Similar threads
Similar threads
| Thread starter | Title | Forum | Replies | Date |
|---|---|---|---|---|
| D | Howto enable external ssh access to netgear r7800 (i have latest Voxel & Kamoj installed) | NETGEAR AC Wireless (Wi-Fi 5) | 2 |
Similar threads
Latest threads
-
Proxmox running VM of Ubuntu and a SMB server how to make SMB multichannel work?
- Started by DJones
- Replies: 2
-
Question about locking a port forward rule to a source IP
- Started by road hazard
- Replies: 1
-
-
-
Sign Up For SNBForums Daily Digest
Get an update of what's new every day delivered to your mailbox. Sign up here!