Entware Entware Server mirror failures

[thelonelycoder]

I wanted to build a logic into amtm so that when the selected Entware server fails, it tests the other two servers in the list.
However, I'm having a hard time with the China server, it fails to download the required file although the ping works.

Can anyone confirm:

ping -c2 -W3 mirrors.bfsu.edu.cn

If that is successful, try wgetting the required update file, the architecture does not matter, here we use armv7sf-k3.2

wget https://mirrors.bfsu.edu.cn/armv7sf-k3.2/Packages.gz

This Error 404's out for me in Switzerland. Skynet does not block it for me.

 

[visortgw]

I wanted to build a logic into amtm so that when the selected Entware server fails, it tests the other two servers in the list.
However, I'm having a hard time with the China server, it fails to download the required file although the ping works.

Can anyone confirm:

ping -c2 -W3 mirrors.bfsu.edu.cn

If that is successful, try wgetting the required update file, the architecture does not matter, here we use armv7sf-k3.2

wget https://mirrors.bfsu.edu.cn/armv7sf-k3.2/Packages.gz

This Error 404's out for me in Switzerland. Skynet does not block it for me.

Same here in US...

# ping -c2 -W3 mirrors.bfsu.edu.cn
PING mirrors.bfsu.edu.cn (39.155.141.16): 56 data bytes
64 bytes from 39.155.141.16: seq=0 ttl=42 time=291.813 ms
64 bytes from 39.155.141.16: seq=1 ttl=42 time=288.576 ms

--- mirrors.bfsu.edu.cn ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 288.576/290.194/291.813 ms
# wget https://mirrors.bfsu.edu.cn/armv7sf-k3.2/Packages.gz
--2024-01-12 16:10:35--  https://mirrors.bfsu.edu.cn/armv7sf-k3.2/Packages.gz
Resolving mirrors.bfsu.edu.cn... 39.155.141.16, 2409:8700:2482:710::fe55:2840
Connecting to mirrors.bfsu.edu.cn|39.155.141.16|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2024-01-12 16:10:48 ERROR 404: Not Found.

#

 

[thelonelycoder]

They seem to have either a latency or availability problem over there. Occasionally it gets through for me.
I'll leave the test as is, it's just depressing that the main entware server bin.entware.net also seems to have availability problems.
Maybe @ryzhov_al has an explanation for the recent mysteries reported in other threads.

My mirror entware.diversion.ch runs fine as far as I can tell but may be outdated for a little while when Entware packages get updated by the Entware team.
I have to run the mirroring manually due to a lack of cron support on my current diversion.ch server.

 

[Ripshod]

Download fails with the same 404 error here in the uk too.

 

[Viktor Jaep]

Probably due to the "Great Firewall"

 

[Viktor Jaep]

They seem to have either a latency or availability problem over there. Occasionally it gets through for me.
I'll leave the test as is, it's just depressing that the main entware server bin.entware.net also seems to have availability problems.
Maybe @ryzhov_al has an explanation for the recent mysteries reported in other threads.

My mirror entware.diversion.ch runs fine as far as I can tell but may be outdated for a little while when Entware packages get updated by the Entware team.
I have to run the mirroring manually due to a lack of cron support on my current diversion.ch server.

I make no claim knowing what kind of space or functionality requirements there are to host a copy of the Entware collective... but would it be possible to upload everything out to a github account, and have it available out there? If I'm not mistaken, there's some automation functionality out there that might help make it easier to keep a mirrored copy of the files in sync? Others are hosting huge blacklist files and such and updating them with automation... you'd think it would be possible for a project like this to host their files out there? Or SourceForge? Like what RMerlin uses? Has all this already been addressed and/or debunked?

 

[thelonelycoder]

I make no claim knowing what kind of space or functionality requirements there are to host a copy of the Entware collective... but would it be possible to upload everything out to a github account, and have it available out there? If I'm not mistaken, there's some automation functionality out there that might help make it easier to keep a mirrored copy of the files in sync? Others are hosting huge blacklist files and such and updating them with automation... you'd think it would be possible for a project like this to host their files out there? Or SourceForge? Like what RMerlin uses? Has all this already been addressed and/or debunked?

If you have a mirror script for me that simply works with the Entware server, then that'll do.
The main Entware team server and one mirror (mine). then I would ditch that Great Firewall protected mirror. In a heartbeat.
There will be a reason why the files themselves are not hosted on GitHub by the Entware team, just the repo.

 

[thelonelycoder]

Also, remember that only Asuswrt-Merlin routers use the two mirrors with my options provided in amtm. All other devices like NASes, OpwnWRT and so on routers only use the main Entware server. The load might be tremendous for all the architectures.

 

[Viktor Jaep]

If you have a mirror script for me that simply works with the Entware server, then that'll do.
The main Entware team server and one mirror (mine). then I would ditch that Great Firewall protected mirror. In a heartbeat.
There will be a reason why the files themselves are not hosted on GitHub by the Entware team, just the repo.

Like this? Apparently wget has a mirror command...

wget -mkxKE -e robots=off https://bin.entware.net/

EDIT: just tried this out, and seems to be downloading the entire https://bin.entware.net contents to my local drive... Not sure if on the next round, it would speed up and skip files that are identical, so that it only updates changes? We shall see I guess!

EDIT2: It took about 2hrs to download all main directories and their most current version files. It then started on the \archive directories, and my goodness... there are a TON of files in these. I have a feeling it would take another 6-8hrs to completely download all the other \archive directories for each directory set.

I also tried running the command which should only download newer version of files... but it's hard to tell if it will work, until they post a new version of something out there:

wget -N https://bin.entware.net/

 

[JGrana]

Like this? Apparently wget has a mirror command...

wget -mkxKE -e robots=off https://bin.entware.net/

EDIT: just tried this out, and seems to be downloading the entire https://bin.entware.net contents to my local drive... Not sure if on the next round, it would speed up and skip files that are identical, so that it only updates changes? We shall see I guess!

EDIT2: It took about 2hrs to download all main directories and their most current version files. It then started on the \archive directories, and my goodness... there are a TON of files in these. I have a feeling it would take another 6-8hrs to completely download all the other \archive directories for each directory set.

I also tried running the command which should only download newer version of files... but it's hard to tell if it will work, until they post a new version of something out there:

wget -N https://bin.entware.net/

Your a brave soul! That’s why we love you @Viktor Jaep

 

[Viktor Jaep]

Your a brave soul! That’s why we love you @Viktor Jaep

Apologies if access to https://bin.entware.net was hit or miss because I was consuming all its bandwidth. LOL I've stopped the test.

 

[thelonelycoder]

EDIT: just tried this out, and seems to be downloading the entire https://bin.entware.net contents to my local drive... Not sure if on the next round, it would speed up and skip files that are identical, so that it only updates changes? We shall see I guess!

I should have warned you. There's a lot of folders/files I don't download.

Apologies if access to https://bin.entware.net was hit or miss because I was consuming all its bandwidth. LOL I've stopped the test.

I use something sensible as --limit-rate=5m and these arguments and some more:

wget --timestamping --no-check-certificate --tries=10 --timeout=100 --limit-rate=5m --continue

 

[ryzhov_al]

They seem to have either a latency or availability problem over there. Occasionally it gets through for me.
I'll leave the test as is, it's just depressing that the main entware server bin.entware.net also seems to have availability problems.
Maybe @ryzhov_al has an explanation for the recent mysteries reported in other threads.

Unfortunately, no evidence of traffic troubles on our side. Stable hosting on one of largest Russian cloud, no any geo/ip/bandwidth/traffic restrictions.

I suspect some connectivity issues due to trade/sanction restriction between some countries. If it's occasional, just poke me on IRC, I`ll check that in real time.

 

[thelonelycoder]

Unfortunately, no evidence of traffic troubles on our side. Stable hosting on one of largest Russian cloud, no any geo/ip/bandwidth/traffic restrictions.

I suspect some connectivity issues due to trade/sanction restriction between some countries. If it's occasional, just poke me on IRC, I`ll check check that in real time.

Thanks Alexander for your input.

 

[powersupply]

tired to install past day, get this error.
Tried different connections etc
Is it just me or are the links all really down??
Thanks
I'm in the US btw, can acces those via http fine....

 

[thelonelycoder]

The links are not all down. You either have a network problem or your routers firmware is hopelessly outdated and the CA certificates are no longer valid.

 

[powersupply]

Good morning!
Thanks for your reply.
I *LOVE* all this AMTM/Diversion stuff, barely check webpages on the go anymore, hate those constant ad pictures and videos popping up all the time, so thank you for this.

Checked network thoroughly, all ok as I can tell.
Router is RT-AC68 don't know about CA certificates. The USB uSD card died, so here I am.
How do I update CAs please?

 

[powersupply]

as mentioned, I can reach links via browser...
Found certificate 2033/10/22 but that's for the browser?
Updated and rebooted, ep still not connecting, sorry.

 

[thelonelycoder]

as mentioned, I can reach links via browser...
Found certificate 2033/10/22 but that's for the browser?
Updated and rebooted, ep still not connecting, sorry.

Check the firmware version your router runs on. You see that in the routers WebUI. If the version is not 386.12_4, then download that firmware here: https://sourceforge.net/projects/asuswrt-merlin/files/RT-AC68U/Release/
Unzip the file on your PC and then go to the routers Administration and select the tab where you can update the firmware.
Select the .trx file in the unzipped folder and let it run the update.

Don‘t come back asking unless you are on the latest version.

 

[dave14305]

Don‘t come back asking unless you are on the latest version.

This could end up being a modded T-Mobile router that got “stuck” on an older firmware version due to Asus’ safeguards. Can’t wait for the ending!